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ABSTRACT 

This paper examines the balance between autonomy and 
accountability in education from the perspective of Massachusetts and Texas 
charter schools and their state authorizers. By analyzing national survey 
data, interviews with key policymakers, and case studies of six low, medium, 
and high performing schools, the paper examines the organizational needs of 
charter schools and how government oversight policies address or mitigate 
those needs. The paper draws lessons from the charter school governance model 
and applies them to the efforts of districts moving toward more decentralized 
governance systems such as charter districts. It builds on Hassel's 1999 
work, "The Charter School Challenge, " which highlights three strategies in 
which charter schools might achieve systemic reform (the laboratory, 
competition, and replacement theses) . This paper notes a fourth thesis, the 
governance thesis, which hypothesizes that the primary benefit of charter 
schools is in changing the basic district structure so that schools have the 
opportunity to control their fiscal and human resources while the central 
government holds them accountable for performance. (Contains 123 references.) 
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EXECUTIVE SUMMARY 



In a time when states and districts are pushing for more accountability in exchange for 
increased local control, charter schools serve as a helpful case study in examining the 
appropriate role of government in public schooling. This paper looks at the basic bargain 
of autonomy for accountability from the bottom-up and the top-down - from the 
perspective of charter schools and their respective state authorizers in Massachusetts 
and Texas. Through an analysis of national survey data, interviews of key policymakers 
in Massachusetts and Texas, and in-depth case studies of six schools (a low, medium, 
and high performing school in each state), I examine the organizational needs of charter 
schools and how government oversight policies serve to address or mitigate those needs. 
The larger purpose of this work is to draw the lessons from the charter school 
governance model and apply them to the efforts of districts moving toward more 
decentralized governance systems such as charter districts. 

The Governance Thesis 

My work builds on Bryan HasseTs work. The Charter School Challenge (1999), in which he 
postulated that there were three overarching strategies in which charter schools might 
lead to systemic reform: 

1) the laboratory thesis that suggests charter schools, freed from bureaucratic 
constraints, will develop innovative approaches to education and by sharing 
those ideas, will improve the system at large. 

2) the competition thesis which posits that districts will offer improved services in 
order to avoid the financial losses associated with students leaving the system. 

3) the replacement thesis that states that the district structure will become so calcified 
and inefficient that it will collapse under its own weight and that all schools will 
eventually become charter schools. 

I suggest that there is a fourth thesis, what I call the governance thesis, which 
hypothesizes that the primary benefit of charter schools will be in changing the basic 
district structure so that schools are given the opportunity to control their fiscal and 
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human resources while the central government holds them accountable for performance. 
What can policymakers learn from the charter school movement in Massachusetts and 
Texas thus far? 

Autonomy: Key Findings 

A primary lesson is that autonomy is a powerful opportunity that comes with significant 
responsibilities. School-level decision-making was essential iii inspiring and enabling 
the educational entrepreneurs in my sample to create exciting schools. However, this 
independence came with the reality of finding and funding a facility as well as 
managing the elaborate administrative and governance demands of running a public 
start-up organization. As a result, in some cases, the responsibilities associated with 
high levels of autonomy threatened to impinge on the core functions of schooling — 
teaching and learning — if the schools did not have the internal capacity to manage their 
new found freedoms. Therefore, my work suggests that the benefits of autonomy 
should be considered in light of a school's capacity (or the authorizer's ability to build 
that school's capacity) to manage this new level of responsibility. That is, I found that 
increased autonomy was not always beneficial to improving school performance — 
autonomy may have enabled excellent schools to thrive, but it did not ensure it. 

Accountability: Key Findings 

On the accountability side of the ledger, charter school initiatives have made strides in 
redefining the culture of public school accountability and they have engaged in the 
political process of defining accountability. However, several challenges still lie ahead. 
Building on the analytic frame of Hill and Celio (1998), I examine three critical tradeoffs 
chartering authorities must confront: a) how to provide the flexibility or opportunity that 
schools need to excel without neglecting the responsibilities incumbent upon a public 
agent; b) how to provide the necessary support or capacity to failing schools with limited 
funds; and c) how to create incentive systems that inspire increased performance without 
unduly limiting the autonomy upon which these schools are founded. 

In light of this framework, I found that: 
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1) Charter school authorizers in Massachusetts and Texas were providing a 
significant amount of opportunity or autonomy to the schools within their 
purview, but I also found that some schools saw that level of freedom 
diminishing over time as government reassumed control. 

2) The facility, start-up, and administrative costs of creating an independent public 
school generally seemed to exceed the capacity available at the school level or the 
level of support available through the state authorizer. 

3) Incentives should be built on a mix of comparable and customized measures. 
That is, the state testing systems in these two states provided valuable and 
reliable data to external stakeholders and encouraged the lowest performers to 
improve. However, the customized site visit process, when done well, was able 
to inspire schools at all performance levels - low, medium and high - to 
improve. 

This thesis provides a window into the basic bargain of autonomy for accountability that 
has been struck between Massachusetts and Texas charter schools and their respective 
authorizers. In so doing, it offers lessons for U.S. policymakers interested in moving 
toward more decentralized systems of public education. 
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Chapter 1: Introduction 



Not only will [accountability] make or break the charter ''movement" itself, it will also be 
the primary source of evidence as to whether that movement is making a valuable 
contribution to the improved education of American children., .or is another half-tried 
reform fad that sinks into the sand like so many others. (Manno, Finn, Bierlein, & 
Vanourek 1997, p. 14) 

OVERVIEW 

"Autonomy for accountability" — that is the basic bargain that is echoing in the halls of 
state houses and school district offices across the United States. For example, 
Superintendent Thomas W. Payzant's vision for the Boston Public Schools was built on 
this bargain. He wrote. 

Challenging expectations for all students and employees should be established 
and results monitored centrally, but individual schools should be given 
flexibility through shared decision-making to determine how best to use their 
resources to meet district standards. The quid pro quo for school autonomy is 
accountability for results (Payzant, 1995). 

There is bipartisan support for the belief that some iteration of this basic bargain will fix 
our nation's public schools. Hugh B. Price, president of the National Urban League, 
advocates for all urban schools to become charter schools, stating that he wants to 
"liberate them from the stifling central-office bureaucracy and give them the latitude to 
operate the way independent secular schools do" (Olson, 2000, p. 26). Both Democrats 
and Republicans support this push toward local control in exchange for results. For 
example. President Bush's proposal for the federal role in public education, "No Child 
Left Behind," is built on this foundation.^ The plan states: 

Accountability for student performance must be accompanied by local control 
and flexibility. If schools are to be held to high standards, they must have the 
freedom to meet those standards. (USDOE, 2001, p. 7) 



ERIC 



1 

Herdman 



8 



But what does this rhetoric really mean, and how will it improve public school 
performance? What are the basic tradeoffs in creating systems of schools built on this 
basic bargain? In attempting to answer these questions, I examine the relationship 
between charter schools and their government oversight agencies, which I call 
authorizers?- Charter schools, as "this country's most aggressive version of school 
decentralization" (Odden & Busch, 1998, p. 48), may represent the purest form of this 
quid pro quo of autonomy for accountability. Unlike traditional public schools, charter 
schools are meant to have less bureaucratic regulation and more control over their 
resources in order to generate new ideas and produce better results. In exchange for 
these freedoms, charter schools face high-stakes consequences if they do not perform to 
expectations. They can be shut down if they fail to meet the goals set out in their 
"charter" with an authorizer or do not attract enough "customers" (i.e., students) to 
meet their expenses. 

At their core, charter schools are independent public schools of choice. They are 
independent in that they can control their own resources, public in that they are free and 
open to all comers, and schools of choice in that the students, parents, and faculty choose a 
school, rather than being assigned to it. 

The charter concept has been one of the fastest growing ideas in education reform over 
the last decade. The first charter school law was enacted in 1991 in Minnesota, and in 
the ten years since, 38 charter school laws have been enacted and approximately 2,153 
charter schools have become operational (Center for Education Reform website, October, 
2001).3 

Within this broad charter school definition there is wide variation by state. This paper 
examines the charter initiatives in Massachusetts and Texas. These two states provide 
substantial autonomy and have nationally recognized accountability systems. Within 
these states, I examine the role of state agencies in holding charter schools accountable. 
This relationship between the authorizer and a charter school is important to study 
because optimally, charter school systems are meant to serve as models of 
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accountability .4 As Kate Zernike (2001) reports in the New York Times: "Charter schools 
were to be more than good schools; they were to teach the country how to hold all 
schools responsible for their performance." (p.3) 

As will be explored in more depth throughout this thesis, implementing this rhetoric has 
proven far more difficult than charter proponents may have imagined. In fact, Richard 
Elmore of Harvard University argues that the states have largely failed in their efforts to 
hold charter schools accountable for performance. He states: 

The states have blown it on accountability .. ..They've chosen the path they've 
chosen since the 19^*^ century - they can't regulate anything except school 
mismanagement, everything else is judgement and they don't want to make 
judgements on the quality of the instructional program. (Elmore as quoted in 
Zernike, 2001, p. 3) 

The data thus far tends to support Elmore's assessment. As of 2001, only 4% (86 of 
2,153) of charter schools have been closed, and of those, 79 have closed for financial 
problems or malfeasance. Just seven (or 0.3%) have been closed for poor academic 
performance alone (Center for Education Reform, 2001). ^ 

While charter school authorizers have not yet solved this puzzle, I would argue that it is 
premature to classify the charter school governance model as a failed experiment. As 
Elmore rightly points out, government agencies are not experienced in making 
subjective judgements on school performance, and since the average charter school is 
just three years old (USDOE, 2000), many authorizers are facing this challenge for the 
first time. This paper examines charter schools as a case study in public school 
oversight. 

It examines this problem from the school and the authorizer perspectives. First, through 
survey data and case studies I identify the key challenges charter schools face in 
building their capacity. Second, I explore the mechanics of the oversight systems in 
Massachusetts and Texas to dissect whether these school-level needs are being met and 
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how schools respond to these policies. I argue that this charter school governance 
model, while far from perfect, has many lessons to share with districts moving toward 
more decentralized models. Further, it begins to unravel why these oversight decisions 
are so complex and to discuss the key design principles policymakers should consider 
when crafting such policies. 

Charter schools and the general public have a stake in understanding this process. In 
order to know how to best allocate resources, charter schools need to be working within 
a policy framework that is transparent. Potential charter school funders and lenders also 
need to understand the authorizer's expectations so that they can make decisions about 
whether to grant or lend resources to a given institution (Hassel, 1999). From the 
perspective of citizens, "until charter granting agencies have policies in place that define 
charter schools' accountability for results, taxpayers can not feel confident that the 
bargain for autonomy for accountability is working" (Hassel, 1999, p. 160). Moreover, 
this issue is critical for policymakers because charter schools are part of a larger U.S. 
trend that encourages local control in exchange for meeting central performance 
standards. As policymakers move further down this path, an examination of charter 
school oversight policies may offer lessons for state or district agencies attempting to 
redesign how all public schools are governed. 

Research Questions 

Through an examination of policymakers and schools in Massachusetts and Texas, my 
analysis will explore the following questions: 

• Whflf are the organizational needs of charter schools? 

• What tradeoffs does government face in designing governance systems to improve charter 
school performance? 

• What can charter schools teach us about the governance of public education? 

The remainder of this chapter will provide a summary of the methods used in 
conducting the study and a roadmap for the thesis. 

METHODS 

er|c 
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My aim in this thesis is to examine the charter school model as a vehicle for exploring 
how public schools should be governed in the United States. 

This study compares and contrasts the charter school accountability policies in 
Massachusetts and Texas and analyzes how charter schools in those states responded to 
their respective policies. It will not explore whether the accountability policies of charter 
schools yield better performance than district schools. The data w'ere gathered from. 
December 1999 to June 2001. 

Data Collection 

To understand the state perspective, I interviewed a total of 20 lawmakers and state 
agency representatives in Texas and Massachusetts. I also conducted a review of local 
media archives, state policy documents, and the legislative records in each state from 
two years prior to the passage of the laws to the present, i.e., from 1991 to 2001 for 
Massachusetts and from 1993 to 2001 for Texas. 

To understand the view from the schoolhouse, I gathered information in three ways. 
First, the National Charter School Study, Fourth Year Report (USDOE, 2000) provided 
survey data of charter school leaders from across the United States on their 
accountability relationships with their authorizers.^ Second, I explored questions raised 
in the survey data through case studies of six schools. I chose a high-, low-, and 
medium-performing school in both Texas and Massachusetts (as measured by their 
respective state tests, the MCAS and TAAS). I interviewed a total of 52 members of 
these school communities. Third, I also reviewed state test scores, charter school 
applications, armual reports, and site visit reports from the time the schools were 
granted charters until their five-year renewal decisions. 

Choosing Massachusetts and Texas 

Massachusetts and Texas were natural choices of states to study for three reasons. First, 
because I wanted to study the charter renewal process at the end of year five, I chose 
states that had enacted laws before 1996 so that I could study the charter renewal 
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process in 2001. Since Massachusetts had enacted its law in 1993 and Texas had enacted 
its law in 1995, these two states met this general criterion. 

Second, I wanted to study two states that had well-established accountability systems. 
National Assessment of Educational Progress (NAEP) scores, the "nation's report card," 
for students in both Texas and Massachusetts were relatively high (e.g., when 
comparing fourth and eight graders' scores nationally, both state placed among the top 
ten states).^ Texas' accountability system, which is built upon the Texas Assessment of 
Academic Skills or TAAS, is credited with gains among majority and minority students 
in math and reading that were so dramatic they were dubbed the "Texas Miracle" 
(Grissmer et al., 2000). Several researchers have argued that TAAS has had deleterious 
effects on students of color,® and that the gains have been inflated (Klein et al., 2000). 
However, despite these concerns, Texas represents one of the few states in the United 
States with a well-developed accountability system and a relatively mature charter 
school initiative. 

Massachusetts' charter school accountability system has not yet been linked to improved 
academic performance, but it is recognized as one of the most thoughtful policies in the 
country. Finn, Manno and Vanourek (2000) describe the Massachusetts accountability 
system this way: 

Massachusetts... has been termed the "Harvard" of charter authorizers due to its 
deliberate pace and selectivity... Massachusetts is widely known for having the 
most meticulous charter accountability system in the country, (p. 129) 

Third, these two states not only had strong accountability systems, but charter schools in 
these states also had high levels of autonomy (USDOE, 2000; Hill et al., 2000). In a study 
of the nine states that had mature charter school initiatives in 1996, Massachusetts and 
Texas had the same high levels of local control.^ Texas and Massachusetts charter school 
leaders reported having the highest levels of control over their curriculum (both 
reported 100% control), hiring (both reported 100% control) and budget (Massachusetts 
charter leaders reported 92% control and Texas charter school leaders reported 86% 
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control) (Figure lA, below). In contrast, charter leaders in Georgia, the lowest ranking 
in school-level autonomy among the nine states, reported levels of control between that 
were 23% and 29% lower in each of these three categories (Figure 1 A). This perception 
of autonomy is important because it indicates that the policy environments in 
Massachusetts and Texas provide charter school leaders with sufficient flexibility to 
respond to their local demands. 
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Figure lA. Relative Autonomy in Massachusetts, Texas and Georgia. Among the nine 
states in the Hill et al., (1999) study, Texas and Massachusetts charter school leaders 
reported the highest levels of autonomy and Georgia charter school leaders reported the 
lowest. 




□ Curricular 
■ Staff 

□ Fiscal 



Georgia Texas Massachusetts 



Source: USDOE, 2000. Disaggregation of curricular, staff and fiscal control done by Celio 

(2000, p. 16.). 

This combination of autonomy and accountability is not universal to all charter school 
initiatives. Based on case studies of six states (Arizona, California, Colorado, 
Massachusetts, Michigan, and Georgia), Hill et al. (1999) developed a four-part typology 
characterizing how authorizers across the United States relate to charter schools: 

• ambivalent about approving charters and conduct only minimal oversight; 

• reluctant to approve charters and conduct aggressive compliance-based oversight; 

• willing to approve charters, and conduct balanced performance- and compliance- 
oriented oversight; or 
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• enthusiastic about approving charters and conduct minimal oversight. 

The distinctions among these broad types become clearer by using a matrix of autonomy 
vs. accountability (see Figure IB). In this typology, California would represent Type 1. 
Its local school boards are virtually required to grant charters by California state law 
since an applicant rejected by the local board can go to the county and then the state to 
appeal the process. As a result, the districts with no incentive or funds to oversee 
charter schools provide little oversight and offer the charter schools limited autonomy. 
Georgia exemplifies Type 2. Its law requires district and teacher sign-off at all levels of 
charter oversight. As a result, few charters are granted, and those that are are highly 
regulated. The Massachusetts and Texas charter school accountability policies 
exemplify Type 3.^® They provide reasonable oversight and high levels of autonomy. 
And, finally, Arizona represents Type 4. Arizona has 416 charter school sites, more 
charter schools than any other state (Center for Education Reform website, October, 
2001). Its law provides maximum autonomy for these schools, but less stringent 
oversight. 



Figure IB. Autonomy vs. Accountability Two-by-Two. This table summarizes four 
different types of charter school authorizers in the United States. The numbers in the 
top left-hand corner of each cell correspond to the typology developed by Hill et al., 
1999. 



High 

>> 

E 

o 

c 

o 

< Low 

"5 

o 

o 



External Accountability 



High 


Low 


3 


4 


High Accountability/ 


Low Accountability/ 


High Autonomy 


High Autonomy 


[Massachusetts/T exas] 


[Arizona] 


2 


1 


High Accountability/ 


Low Accountability / 


Low Autonomy 


Low Autonomy 


[Georgia] 


[California] 



9 

Herdman 



Another way to understand the distinctions between the states is to examine their state 
laws. Table lA demonstrates that Arizona and California provide rninimal restrictions 
on the source and number of charter schools, but that they do not have well-established 
state testing programs.^^ Georgia has a state testing program and offers unlimited 
numbers of charters, but since those charters are not legally independent from the 
district, their autonomy is limited. Massachusetts and Texas both offer significant 
autonomy and accountability, however, Massachusetts has more restrictions on its 
growth. 



Table lA. Legal Comparison of Five States. This table summarizes some of the 
accountability and autonomy provisions in the five states listed. 
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X 
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X 


X 


X 


X 



Sources: Center for Education Reform (2001) and Goertz et. al. (2001). 



In sum, few charter states provide a balance of autonomy and accountability that might 
lead to the growth of a school's organizational capacity. I chose to study Massachusetts 
and Texas because they appear to provide that balance. 



Choosing Schools 

I visited six schools, three in each state. For each school, I reviewed reports on the 
performance of the schools, and conducted a newspaper search for the year prior to the 
school's opening to the present (five years in Texas and six years in Massachusetts) to 
gain a sense of the political and historical context surrounding each school. I also 
examined all available correspondence from the state relative to each school's 
performance (annual reports and site visit reports) from 1996 to 2001. 

Er|c 



10 

Herdman 



17 



In addition to extensive document reviews of each school, I visited each school and 
attempted to interview the school leader, teachers, parents, and board members (and a 
representative of the educational management company, if applicable). I conducted 
day-long visits at each school, and each interview lasted approximately 45 minutes. 
(Please see Attachment 1 for protocols.) 

The criteria for choosing the schools is described in more detail in Attachment 1, but in 
short, these schools had the following attributes in common: 

♦ All the schools were new start-ups 

♦ All the schools were in operation for four or more years 

♦ The schools represented a range of performance, and 

♦ The schools are all sponsored by the state (as opposed to a district, university or 
some other authorizer). 

To summarize, this paper is built on the premise that public school governance is 
moving toward systems in which central standards and decentralized control of school- 
level decisions will be the norm. Given this premise, I examined charter school 
governance policies, policies founded on this basic design principle of autonomy for 
accountability, in the hope that they might serve as models for oversight. In particular, I 
took a closer look at Massachusetts and Texas because, based on national research, these 
represented two of the few states that offered a balance of autonomy and 
accountability — a balance that should, theoretically, lead to increased student 
performance (Hill et al., 1999). This study examines the construction of these state 
policies and how charter schools responded to those policy frameworks. My analysis 
looks specifically at the costs and benefits associated with three core policy components: 
incentives, opportunity, and capacity. 

ROADMAP FOR THIS THESIS 

This paper is divided into seven chapters. It represents a distillation of my dissertation. 
Unraveling the Basic Bargain: A Study of Charter School Accountability in Massachusetts and 
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Texas. The logic behind its organization is as follows. Chapter 2 begins with a 
discussion of the literature and theory regarding charter school accountability to put this 
work in context. 

Chapter 3 describes a working theory of charter school accountability built on the 
conceptual frame of incentive, opportunity, and capacity developed by Paul Hill and Mary 
Beth Celio (1998). Schools need incentives to overcome the inertia of mediocrity; 
opportunity or flexibility to respond to the incentives appropriately; and human and 
fiscal capacity to respond to the incentives and opportunity available to them. This 
frame redefines "accountability." Rather than a punitive, "gotcha" approach for failing 
schools, accountability in this light is seen as a system to improve schools. In particular, 
it is seen as a vehicle to improve organizational capacity. This conceptual frame 
acknowledges the need for external pressures to perform, but it places equal weight on 
the need to build and nurture the capacity of schools from within. 

Chapter 4 examines the autonomy side of the basic bargain, i.e., it analyzes the 
organizational needs of these independent public schools in an effort to understand 
what accountability systems should be designed to fix. The chapter begins with a 
summary of national survey data on the key barriers to implementation that charter 
school leaders have encountered. With the survey data as a context, I look more closely 
at six charter schools (three per state). These challenges range from the need for better 
facilities to the need to better train and retain teachers; from the need to manage heavy 
administrative demands, to the challenge of inconsistent leadership. 

Chapter 5 begins to look at the accountability side of the ledger, by examining the capacity 
and opportunities within the Massachusetts and Texas oversight systems. I argue that 
while opportunity or local autonomy enables school to excel, it does not ensure it. 
Further, I examine the limitations of authorizers in addressing charter school needs and 
map out the array of private and public players that play this capacity-building role. 
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Chapter 6 examines the incentive structures in each state and summarizes the political 
processes in developing the charter school laws and regulations in Massachusetts and 
Texas, It describes how authorizers crafted those laws into oversight policies and how 
those policies impacted charter school needs. Further, it explores questions such as: how 
did the goal setting process at each school actually work and what effect did test scores 
versus site visits have on the governance and instruction of these schools? It also 
explores the challenges of charter renewal through an in-depth study of a school that 
was in the gray area between charter renewal and charter revocation. 

Chapter 7 provides a discussion of the lessons learned and offers implications for 
policymakers. Implications for this work extend beyond the charter school arena. As 
state and federal governments continue to build policies around the quid pro quo of 
autonomy for accountability, it will become increasingly important to understand how 
to hold schools with differentiated missions accountable for performance. More 
importantly, policymakers may find this thesis helpful in exploring how government 
can use accountability policies not only to passively monitor performance, but also to 
actively encourage systemic improvement. 



Endnotes 



^ The Democratic counter-proposal by Senators Lieberman and Bayh also calls for testing at 
multiple grade levels and sanctions if a school does not perform to expectation after three years. 

A primary difference between the two plans is that the Bush proposal called for giving parents in 
under-performing schools $1,500 in the form of vouchers, while the Lieberman-Bayh proposal 
called for dedicating those funds to supporting charter schools. 

2 The types of agencies that may authorize charter schools vary from state to state. For example, 
in Massachusetts the State Board of Education is the only authorizer, but in Texas, the State Board 
of Education and districts can serve as authorizers. In other states, such as Michigan or Indiana, 
universities or non-profit organizations can authorize charters as well. 

^ The number of charter campuses is significantly higher in that many states allow a charter 
holder to open multiple campuses. 

^ As Ted Kolderie (1990), a pioneer of the national charter initiative wrote, the intent is not about 
creating a few good schools, but to ''create dynamics that will cause the main-line system to 
change so as to improve the education for all students" (1). 

5 Richard Roths tien (1998) argues that no charter schools have been closed for academic reasons. 

^ These data reflect the opinions of charter school leaders in the nine states that had charter 
schools in operation for two or more years in the spring of 1998. There were 373 charter schools 
in the universe of interest, and of these, 294 responded (79% response rate). In Massachusetts 
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and Texas, the response rates were 75% and 82%, respectively. The Center on Reinventing Public 
Education (with input from the author) developed the survey questions relating to charter school 
accountability and incorporated those into the USDOE survey since both RPP and CRPE were 
contractees of the USDOE. 

^ NAEP rankings in 1996 and 1998 for Massachusetts and Texas fourth and eighth grade students: 
In Massachusetts, grade 4 students were ranked ninth nationally in 1996. In 1998, Massachusetts 
grade 8 students were ranked fifth in reading and second in writing. Similarly, in 1996, grade 4 
students in Texas were ranked sixth nationally in math and grade 8 students were ranked tenth 
in math. In 1998, grade 8 students were ranked fourth in writing. 

® See Orfield and Wald (2000) for a review of the research on Texas' testing system. 

9 Among the nine states in the Hill et al. (1999) study, the rank order of the states from the lowest 
reported level of charter school autonomy to the highest, were: Georgia, Wisconsin, California, 
Arizona, Michigan, Colorado, Minnesota, Texas, and Massachusetts. 

Several other authorizers were cited as Type 3: Chicago; Washington, DC, Charter School 
Board; and some districts in Colorado and Michigan. I did not choose the former two because 
they did not participate in the USDOE national survey of charter school accountability and I did 
not choose the latter two because the state is not the primary authorizer in either state (i.e., 
districts are the primary authorizers in Colorado and universities are the primary authorizers in 
Michigan). 

Both Arizona and California have administered several state tests in the last five years, but the 
testing systems have been changed so frequently, they are not well-established. 
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Chapter 2: A Review of the Literature 



This chapter provides a literature review of what we know about the trajectory and 
intents of the charter school movement. In general, this overview addresses two 
questions: 

• VvTiere did charter schools come from in the United States? 

• How has the initiative grown over time? 

• What are the intents of charter school concept? 

Before summarizing the research to date, it is important to discuss the nature of the 
available literature on charter schools. Few of the early data sources cited below are 
from peer-reviewed journals. Many are from state agencies and may not be 
generalizable beyond that state's unique political context. Some states, such as Texas, 
California, and Michigan, have been studied more extensively than others, creating the 
potential for the experiences in those states to be over-represented (Bulkley, 2000). In 
addition, given the political nature of charter schools, much of the early work was 
sponsored by those with agendas for or against the charter concept. Therefore, where 
possible, I report findings from government sources (which tend to be more neutral) or 
from multiple sources in order to counterbalance potential biases. 

CHARTER ORIGINS 

The charter concept builds on reform efforts of the last 30 years. Magnet schools of the 
1970s were the likely forerunners of current day charter schools (Tyack & Cuban,1995; 
Nathan, 1996). However, the term charter schools did not gain currency until the late 
1980s. Raymond Budde (1988) was the first to write of "charter" districts and schools, 
and A1 Shanker, the late president of the American Federation of Teachers, popularized 
the term in the same year (Shanker, 1988). Around the same time, a Philadelphia school 
district referred to its reform efforts as "chartering," and England gave its schools 
charter-like autonomy by allowing schools to "opt out" of local school jurisdictions via 
Great Britain's 1988 Education Reform Act.^ Despite this apparent confluence of ideas, 
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policies come and go. Why did this one resonate and spread so rapidly across the 
United States? 

The national context in the early 1990s provided fertile ground for the growth of the 
charter school concept. Chubb and Moe's "Politics, Markets and America's Schools" 
(1990) offered three main conclusions that underscore some of the influential thinking at 
that time: 

• Schools do indeed perform better to the extent that they. . .have such general qualities 
as clear goals, an ambitious academic program, strong educational leadership and 
high levels of teacher professionalism. 

• The most important prerequisite for the emergence of effective school characteristics 
is school autonomy, especially from external bureaucratic influence. 

• America's system of public education inhibits the emergence of effective 
organizations. This occurs, most fundamentally, because its institutions of 
democratic control function naturally to limit and undermine school autonomy (p. 
23). 

During this time, the public education system, what Milton Friedman (1979) called an 
"island of socialism in a free market sea" (p. 144), began to change fundamentally .2 
Educational services were beginning to be deregulated and contracted out.^ John 
Donahue (2001), of Harvard's Kennedy School of Government, explains. 

Debates over the private sector's role in education flared with new intensity in 
the late 20* century as an epidemic of anxiety over school performance coincided 
with a surge in legitimacy for private solutions to public problems (p. 1). 

Private companies offered a full range of educational products and services: Sylvan 
Systems provided tutoring in math and English; Public Strategies Group, Inc., provided 
financial and administrative management skills, and Education Alternatives 
Incorporated (also known as Tesseract, which went bankrupt in 2000) offered districts 
complete financial and educational management services.^ David Kearns, former 
chairman and CEO of Xerox and former U.S. Deputy Secretary of Education, led a 
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partnership between business and government to create " break- the-mold schools" with 
the establishment of New American Schools Development Corporation in 1991. Kearns 
said, "Until we find a way to fundamentally change the way we run our schools, we will 
never see long term results" (as quoted in The Education Industry Report, 1996). 

The Politics of Central and Local Control 

Concurrent with this push for decentralization and privatization was an emphasis on 
state-level academic standards. The publication of "A Nation At Risk" in 1983 as well as 
the establishment of the National Council of Teachers of Mathematics (NCTM) 
standards and Goals 2000 in 1989, helped propel the standards movement.^ By 2001, 50 
states had state standards. 

The charter school legislation debate in Massachusetts and Texas in the early to mid- 
1990s was part of two seemingly contradictory policy trends that were gaining 
prominence across the United States. On the one hand, there was a movement afoot to 
decentralize; that is, to move decision-making to the local level via school choice, site- 
based management, and contracting out. On the other hand, there was an equally 
powerful trend to centralize, as evidenced by the standards-based reform movement. 
While these reform efforts may seem antithetical, Susan Fuhrman of the University of 
Pennsylvania notes that this apparent disjuncture was in fact unifying diverse reform 
constituencies, which in turn increased their collective political power. She wrote. 

State curriculum frameworks and reinforcing policies could provide a protective 
structure that would undergird strategies for parental choice and school governance. 
(1994, p.7) 

In short, the timing was right for charter legislation because it blended and advanced the 
concepts of autonomy and accountability.^ 

The first charter school law was enacted in Minnesota in 1991, and the charter concept 
quickly garnered support from Republicans and Democrats throughout the United 
States.^ Many Republicans endorsed charters because they supported smaller 
government and saw charter schools as a potential stepping stone toward the ultimate in 
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local control, school vouchers. Many Democrats supported charter schools because they 
offered the potential to serve as laboratories of innovation for the existing system and 
might serve to co-opt the desire for school choice within a public framework (Hassel, 
2000; Pipho, 1993). As Donahue (2001) points out, 

[T]here are excellent reasons for the breadth of support this movement has 
inspired. Charter schools offer most of the benefits that vouchers can offer, with 
few of the risks. Those who continue to cherish the goals of public education, 
but concede the shortcomings of the status quo, can logically look to charters as 
the most effective friendly threat to conventional public schools. (p.l4) 

Therefore, the charter concept grew quickly because it was a proposal that had support 
from a diverse and broad constituency. 

THE GROWTH OF THE CHARTER MOVEMENT 

More than a decade after the first charter law was enacted in 1991, 37 states, Puerto Rico, 
and the District of Columbia have charter school laws, and over 2,000 charter schools 
enroll approximately 500,000 children in 34 states (as of March 2001; see Center on 
Education Reform (2001) and USDOE, 2000; see also Figure 2A below.)^ 



‘ Wyoming, New Hampshire, and Indiana have charter laws, but do not have any active charter schools as 
of March 2002. 
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Figure 2A. Growth of Charter Schools in the United States. This graph illustrates that the 
number of charter schools has grown at a rate of about 30% or more per year since 1992. 




Source: USDOE (2000) for the data up until 1999. The 2000 data are from the Center on 
Education Reform (www.edreform.com). 

Since the problems in U.S. education are perceived to be worse in our urban areas, it is 
not surprising that 60% of charter schools are located in cities (USDOE, 2000). The 
ethnic and racial profiles of charter school populations generally reflect the 
demographics of the urban centers. As Table 2A summarizes, on average, African- 
American (24% in charter schools versus 17% in all public schools) and Latino students 
(21% in charter schools versus 18%) are generally over-represented in charter schools. 
The percentage of limited English proficient (LEP) students and those eligible for free 
and reduced lunch (FRL) are roughly equivalent to national averages (9.9% of all charter 
school students are LEP, versus 9.8% of all public school students nationwide; 38.7% of 
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all charter school students are eligible for FRL, versus 37.3% of all public school students 
nationally). Table 2A. 

In contrast, special needs students are generally under-represented in charter schools 
(on average). Table 2 A points out that the average enrollment of special needs students 
in charter schools is 8%, versus the national average in all public schools of 11%. 

Various theories have been proffered to explain these lower numbers: 1) that special 
needs students are being dissuaded from enrolling in charters; 2) that parents of 
disabled children are opting to stay in their districts; and 3) that because the student- 
teacher ratios are often lower in charters, students receive more personal attention and 
are thus opting out of their Individual Education Plans (lEPs). 



Table 2A. National Data on Charter School Demographics 
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(1996-97) 


17% 
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59% 


37% 


11% 


10% 



Source: USDOE (2000) 



After nearly a decade of rapid growth, the charter initiative appears to be leveling off 
(Figure 2B). For the first time in eight years (1993-2001), the number of new charter 
schools in 2000-2001 was less than the previous year (Figure 2B). 

Figure 2B. Increase in the number of charter schools per year in the United States. 
The numbers associated with each column represent the number of new charter schools 
granted across the United States from one year to the next. 
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Source: USDOE, 2000 and the Center for Education Reform website (2001). 



When one examines Figure 2B more closely, it is clear that the percentage increases in 
the number of schools per year are slowing as well (Figure 2C). Figure 2C shows a 
steady decline in the annual growth percentage from 1993 to 2001. 

Legislative action is beginning to slow after a decade of rapid growth (Figure 2D and 
Table 2B). The 2000 legislative session was the first since 1991 in which no state passed 
charter legislation.® All this does not suggest that the number of charter schools will 
stop growing; on the contrary, with nearly three-fourths of the states now having charter 
legislation, the number will continue to grow depending on a given state's "cap" or limit 
on the number of allowable charters. However, it does represent an end to the 
exponential growth that marked the early 1990s. It might also signal a shift in focus on 
the part of authorizers from the charter approval process to the charter school oversight 
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process. 



Figure 2C. Number of charter school state laws passed per year from 1990 to 2000. 




Source: USDOE, 2000 
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Table 2B. Charter Laws and Years Passed. This table summarizes the year a given state 



passed charter school legislation. 
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CHARTER SCHOOLS AS A GOVERNANCE MODEL 

What will the charter school legacy be? Has the market pushed the existing education 
system to excel? Freed from bureaucracy's chains, have charter schools generated new 
ideas? Has the charter initiative grown so quickly that it will replace the current district 
structure of governance altogether? 

Bryan Hassel, in The Charter School Challenge (1999), postulated that there were three 
overarching strategies in which charter schools might lead to systemic reform. First, the 
laboratory thesis suggests that charter schools, freed from bureaucratic constraints, will 
develop innovative approaches to education and by sharing those ideas, will improve 
the system at large. Second, the competition thesis posits that districts will offer 
improved services in order to avoid the financial losses associated with students leaving 
the system. Third, the replacement thesis states that the district structure will become so 
calcified and inefficient that it will collapse under its own weight and that all schools 
will eventually become charter schools. 

I suggest that there is a fourth thesis, what I call the governance thesis, in which many 
charter school proponents are now investing their energies.^ This thesis hypothesizes 
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that the primary benefit of charter schools will be in changing the basic district structure 
so that schools are given the opportunity to control their fiscal and human resources 
while the central government holds them accountable for performance. 

In the text that follows, I summarize the rhetoric and empirical data associated with each 
of these four theses. Because each school is shaped by its unique vision and state law, it 
is impossible to make broad claims about all charter schools, but some patterns are 
beginning to emerge. 

Laboratory Thesis 

This thesis builds on the idea that if charter schools had fewer bureaucratic constraints 
they would develop new ideas and disseminate those ideas to other public schools. 

Charter schools are "different" in several ways. One of the biggest differences is their 
size. Charter schools are generally smaller than district schools. The average charter 
school has 137 students compared to a median enrollment of 475 in district-run public 
schools (USDOE, 2000), and they are often organized differently. For example, almost 
half of charter schools are organized in grade configurations that differ from the 
traditional high, middle, and elementary model (USDOE, 2000). They also generally 
have more control over school operations (Arsen et al., 1999 ).^^ However, it is not clear 
that charter schools are generating ideas that are wholly new to public education. 

To the extent that charter schools are generating new ideas, the ideas are "local 
innovations." For example. City on a Hill Charter School in Boston, Massachusetts uses 
"juried-assessments" at the end of each school year to discern how much students have 
learned. Students present their understanding of math and English (perhaps solving an 
algebra problem or interpreting an editorial from the New York Times) in front of small 
groups of professional mathematicians and writers. Certainly, other schools in the state 
may use similar assessments, but this concept was novel among high schools in the city. 
These approaches are common within the broader public school community nationwide, 
but may not be available in a particular district (Finn et al., 2000; Rosenblum Brigham 
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Associates, 1998). Therefore, the dissemination of new ideas may be limited in part 
because there simply may not be much to share. 

The laboratory thesis is also hampered by a lack of vehicles to exchange new ideas that 
do exist. Collaboration between charter and district schools is difficult because there are 
few mechanisms and little incentive to share best practices (Adelman & Firmigan, 1999). 
The sharing of best practices among public schools is difficult under the best of 
circumstances, and, until recently, most states did not dedicate any resources to enable 
charter schools to share what they had learned.^^ In 2000, the U.S. Department of 
Education gave grants to individual charter schools around the country to help 
disseminate best practices, and this funding has given rise to a small number of 
collaborations among charter and district schools in some states, including 
Massachusetts.^2 However, even if there are dedicated resources for collaboration, there 
is an inherent tension between district and charter schools because they are competitors. 
If a district school principal is losing students and resources to a charter school, what 
incentive does she have to share the best practices from her school (Hassel, 2000; Wells 
et al., 1998; Rosenblum Brigham Associates, 1998)? 

Competition Thesis 

This thesis is based on the premise that parental or "market" demands will shape and 
improve public education because schools will change their services to meet consumer 
demands. This thesis builds on the assumptions that a) parents will choose charter 
schools over district schools, and b) the number of charter schools will reach a large 
enough number to force change. 

As market proponents predicted, the data suggest that satisfaction is generally high 
among the parents of charter school students. Seven out of ten charter schools 
nationally have a waiting list (USDOE, 2000), and survey data from several sources 
indicates that parents are generally satisfied with their choices. In a survey conducted 
by Finn et al. (2000), two-thirds of parents said the charter school their child is now 
attending is better than their child's previous school. This finding was consistent with 
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surveys conducted in Texas (Texas Education Agency, 2000) and Minnesota (Center for 
Applied Research and Educational Improvement, 1998)A^ 

However, it is unclear whether strong parent satisfaction will lead schools to improve 
their academic performance. Arsen et al. (1999) argue that parent survey data regarding 
school choice are misleading because asking a parent to score their child's new school is 
in essence asking the parent to evaluate the wisdom of their own choice. Beyond this 
concern, academics do not appear to be the primary criterion by which a parent chooses 
a school; location, hours of operation, and school safety are generally more influential 
factors. Therefore, this raises the question of whether market success is a sufficient 
measure of school effectiveness (Hassel, 1998). 

Assuming parents generally like charter schools, the next question is whether that 
demand has led to changes in district schools. The theory is that if districts lose students 
and their attendant per pupil allotments, then they will change their products and 
services to better meet the needs of their consumers. 

The data indicate that competition has had a limited effect on large districts. Nathan 
(1996), Rosenblum Brigham Associates (1998), and USDOE (2001) have found some 
significant anecdotal changes. For example, in Mesa, Arizona, the district developed 
more alternative public schools and began running advertisements at cinemas in 
response to a high concentration of charter schools in the district. In Boston, "pilot 
schools," schools with charter-like control of their fiscal and human resources, were 
created by the district and local bargaining unit to prevent a small cluster of the more 
innovative schools from opting into the charter system. However, these changes are the 
exception rather than the rule. Due to the lack of facilities, limited start-up funds, and 
statutory limits on the number of charters in many states, it is difficult for any charter 
initiative to gain critical mass in any given district. Only a few states have more than 2% 
of their public school student population in charter schools (Legg Mason, 1999); 
therefore, it is difficult for a given initiative to grow to a size that would actually force a 
district to change.^^ In sum, Eric Rofes' (1997) finding that "the majority of districts had 



26 

Herdman 



gone about business-as-usual and responded to charters slowly and in small ways" (p. 
11) appears to be the norm. 

Replacement Thesis 

This thesis hypothesizes that charter schools will replace district schools altogether. In 
order for this theory to work a) the number of charter schools will need to reach a critical 
mass and, b) there will need to be substantial evidence that this approach is more 
efficient than existing options, i.e., producing better results on the same or fewer public 
dollars. 

For the same reasons stated above - statutory limits, facilities challenges, lack of start-up 
capital - the charter initiative will not likely reach a critical mass in most states. In 
addition, it is not politically or logistically feasible for states to oversee large numbers of 
schools. The United States' historically strong support of local control makes the notion 
of authorizing all schools through a state agency unrealistic. Further, state agencies are 
usually situated near a state's capital and are often hours away from the far reaches of 
any given state. As John Stevens of the Texas Business and Education Collective writes, 
"The State... should not generally be in the business of running public schools" (Stevens, 
1999, p. 4). State higher education institutions also authorize charters in some states, but 
state colleges and universities would encounter the same limitations regarding capacity 
and geography that state agencies do. 

Geography may not appear to be an insurmountable problem because states can use 
regional offices or universities can use satellite campuses to oversee schools, but unless 
those decentralized bodies are given the authority to make high-level decisions, the onus 
of these decisions will still be on state offices. Even if this authority were decentralized, 
the resources and training necessary to train all of the employees at each of the remote 
sites is likely more of an investment than most state legislatures are willing to make. 

The district is the best structure for overseeing charter schools locally, however, as will 
be discussed in more detail below, local school boards and organized labor have been 
resistant to altering the decision making power they now have. In short, it appears 
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unlikely that the current authorizer models have the capacity to replace the district 
structure without wholesale transformations of entrenched power arrangements. 

Even if this state-level governance model could be constructed to manage large numbers 
of schools, it is not clear that charter schools are currently using funds more efficiently 
than district schools. Charter schools offered what Odden and Busch (1998) called "the 
country's most aggressive version of school decentralization" (p. 48). By bypassing 
district central offices, charter schools offered hope that public dollars could be used 
more efficiently, and that by eliminating district-level waste and giving schools direct 
control of their resources, they could respond more efficiently to their community's 
needs. This was important because not only was the performance of US students 
lagging in international comparisons, but Americans were also spending as much or 
more on education than any other industrialized country.^s jn 1998, K-12 public 
schooling was a $320 billion dollar industry in the United States, representing 4% of our 
gross domestic product, (Odden & Busch 1998) In short, pressure was building to 
produce better results with the same, rather than more, public dollars. 

Thus far, there is no empirical evidence to support the idea that charter schools are 
getting proportionally more money into the classroom or producing better results on the 
same public funds. A microfinancial analysis of charter school spending (Herdman and 
Millot, 2000) found that Massachusetts charter schools were dedicating about the same 
percentage of their annual budgets to classroom-related expenses as were district 
schools nationally The reasons for this were, in part, that a) charter schools do not 
have separate facilities funds like districts do, and thus need to draw funds from their 
instructional budgets to address their facilities costs; and b) individual charter schools 
do not benefit from the economies of scale of a district, and therefore, have relatively 
high administrative costs. 

More important, assuming that charter schools' classroom spending is on a par with 
district schools, it does not appear that simply reallocating resource control from the 
district to the schools alone has led to consistently improved performance. The data 
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indicate that the performance among charter school students nationally is decidedly 
mixed. Bulkley (2000) analyzed 21 state-level charter school evaluation reports written 
between 1997 and 2000, and found that there were some gains in Colorado, Los Angeles, 
and Massachusetts, but that the performances in Washington, DC, Michigan, and Texas 
were far less promising.^® 

In sum, it appears the current charter school authorizing structures a) do not have the 
capacity to replace the existing system, and b) have not yet proven that they are any 
more efficient than the existing systems for governing public schools. 

Governance Thesis 

The governance thesis represents a hybrid of Hassel's laboratory and replacement theses. 
It posits that public schools will improve if they are given the autonomy to act locally 
within a district framework of oversight and support. This thesis differs from the 
laboratory thesis by suggesting that the governance model itself is a more important 
innovation than any new ideas that might be derived from individual schools. It is 
distinct from the replacement thesis because it does not argue that the current 
authorizing relationships, in which 57% of charter schools are sponsored by authorizers 
other than a local school board (Center for Education Reform, 2001), will grow to replace 
the existing district structure. Rather, this thesis hypothesizes that if districts adopt an 
appropriate charter-like governance model, schools will improve on a large scale. 

There is a great deal of enthusiasm for decentralizing public education. Ed Kirby (2000), 
a senior administrator in the Massachusetts Charter School Office from 1997 to 1999, 
argues for the power of self-governance. He writes: 

A charter school can make big changes on the spot. It can replace its reading 
curriculum, extend its school year, or fire an ineffective teacher with regard to no 
other rules but its own. It can do any of these things in a matter of days, 
sometimes hours. Charter schools have figured out what researchers and 
legislative commissions still fail to grasp: that the real innovation in charters is 
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nothing more than the power of self-governance - the power to make changes 
that produce results (p. 1). 

However, as stated above, there is no empirical evidence to show that simply changing 
the locus of decisionmaking from a central office to a school will result in improved 
performance. In fact, when decentralization is seen as an end in itself, it has generally 
failed (Malen, Ogawa, & Kranz, 1990; Murphy & Beck, 1994). As Wohlstetter and 
Albers-Morman (1994) point out, if decentralization is to have a chance to be effective, it 
must be linked to a focus on performance. They write 

that deceptively simple change in how schools are managed and governed, as 
attractive as it is to many teachers, principals and parents, turns out to be rather 
meaningless unless it is part of a focused, even passionate, quest for 
improvement. School-based decision-making is one aspect of systemic school 
reform - an approach to improving schools that also includes changing 
instruction, curriculum, [and] the institutional web that surrounds schools to 
achieve an integrated focus on the outcomes of education (p. 1). 

This focus on performance is central to the governance thesis. Over time, the focus of 
schools has become fragmented due to layers of federal and state requirements, along 
with the need to respond to multiple funders. Mary Russo, a principal and director of 
Boston's Annenberg Challenge efforts, called this propensity of schools to get lost in 
responding to multiple demands as "projectitis" (Hill, Campbell & Harvey, 2000). She 
described it this way: 

It's. . .about moving away from "projectitis." In the past you were a good 
administrator if you could pull in as many grants as possible, [but] the problem 
was that you'd end up with a grab bag full of projects and you'd use the money 
for whatever you needed and there was no focus, so we didn't see any 
improvements" (p. 137). 

This next chapter offers a working model for how the "institutional web" described by 
Wohlstetter and Albers-Morman (1994) might be constructed to help schools focus on 
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their mission through an examination of the lessons learned from the state-authorizing 
context that can be instructive for district governance restructuring. 

Endnotes 



1 See Hassel, 1999; Fine, 1994; and Wohlstetter and Anderson, 1994. 

^ Friedman is a Nobel Prize-winning advocate of school vouchers and the author of ''Free to 
Choose" (1979), among other publications. 

3 In the 1970' s, Dade County, Florida was the first U.S. district to experiment with site-based 
management on a large scale. For more on decentralization see Osborne and Gaebler, 1992; 
Bimber, 1993; and Elmore, 1993. 

^ Tyack and Cuban (1995) point out that experiments with the privatization of public education 
began in the late 1960's. For example, they report that in the fall of 1969, in Texarkana, Arkansas, 
that over a hundred students identified as potential drop-outs entered "Rapid Learning Centers" 
set up by Dorsett Educational Systems. See pp. 117-119 for more detail. 

5 Forty-rune states have embarked on initiatives related to academic standards. This is due in 
part to the establishment of the Improving America's Schools Act (lASA) 1994. This law created 
major changes in this $8 billion dollar federal program known as Title I of the Elementary and 
Secondary Education Act. Based on a concern that the operation and impact of Title I over the 
previous 25 years was focused on procedural compliance rather than academic outcomes, the 
new federal approach was built on a state-established performance and content standards, 
accountability for performance, and programmatic flexibility (Geortz and McDuffy, 2001). 

^ Note that efforts to devolve decision-making to the school level in exchange for adherence to 
central standards were happening throughout the world in the 198 O's. See McGinn (1994) for an 
international analysis of decentralization. 

^ In the 2000 Presidential campaign, the Republican candidate, then-Texas Governor George W. 
Bush promised to provide enough federal funding to double the number of charter schools by 
2004. Vice President Albert Gore, the Democratic candidate, promised to continue then-President 
William Clinton's support of charter schools by tripling the number of charter schools by 2004. 

® Indiana passed its charter school legislation on May 2, 2001. 

^ This idea was discussed as a corollary to the laboratory thesis in Hassel (1999). 

An National Education Association study by Koppich, Holmes and Plecki, 1998, found that 
68% of charter school teachers say there is "little or no imion or association involvement in their 
school" (p. iv). 

See Richard Elmore (1996) for a discussion of the challenges associated with large-scale change. 
Also see Bryan Hassel (1999) for a discussion of the challenges faced by New American Schools, a 
private-public partnership, dedicated to large-scale change. 

12 Through the federal Charter School Dissemination Program (Massachusetts funding code 537), 
nine Massachusetts charter schools received $666,325 in FYOl. The grants ranged from $42,085 to 
$75,000. 

1^ Parent satisfaction among Texas charter school parents was not universally positive. Parents of 
students in at-risk charter schools were more satisfied than those with children attending non-at- 
risk charters (TEA, Part Two, 2000). 

1^ Some cities such as Mesa, Arizona and Washington, D.C. do have close to 15% of their students 
enrolled in charter schools. 

15 The Third International Mathematics and Science Study (TIMSS) in 1995 consistently ranked 
U.S. eight grade students behind their peers from other industrialized nations. When compared 
with the United States' five major economic partners (Japan, France, Canada, England, and 
Germany), U.S. math scores placed in the bottom half and science scores fell somewhere in the 
middle (USDOE, 1996). 
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Funding for public education in the US is very uneven across localities because much of the 
funding for public education comes from local taxes. Tax revenues are divided as follows: 47% 
local taxes; 45% state taxes; and 7% federal taxes (Odden and Bush, 1998). 

Howard Nelson et al. (2000) conducted a study of charter school revenues but did not examine 
expenditures. 

1® Bulkley (2000) also notes that the quality of this data were suspect because it was often based 
on less than three years of data, was not student-specific, and did not represent value-added 
scores since baseline data was not generally available. 
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Chapter 3: A Working Theory of Governance 



Focusing on the "governance thesis" described in chapter 2, what should the goals of a 
charter school authorizer be? The primary goal of any government agency overseeing 
public schools is to produce excellence while ensuring equity. However, because state 
and district agencies have had at least 25 years to develop systems of compliance, I will 
focus on the less understood excellence side of the equation. That is, how can 
government oversight systems be designed to promote the systemic improvement of 
charter schools. 

CHARTERS AND THE NEW ACCOUNTABILITY 

In the past, government accountability has been synonymous with compliance 
regulations, i.e., controls on the use of public funds. However, Fuhrman (1999) argues 
that the current trend in public school accountability — what she calls "New 
Accountability" -- deviates from earlier policies in that "states and districts are trying to 
put more emphasis on student performance and less on compliance" (p. 1). Most 
performance-based accountability systems are based on four principles: 

• schools are the primary unit of analysis; 

• student performance is the most important outcome measure; 

• school-level performance is measured against a set of state- or district-established 
standards (usually through some form of standardized test);^ and, 

• incentives in the form of sanctions, interventions, or rewards accompany a school's 
relative performance (Ladd, 1996). 

Differentiating Internal and External Accountability 

The term accountability can take many forms and has many purposes. In broad strokes 
these purposes can be defined in terms of internal and external accountability (Newmann 
et al, 1997; Darling-Hammond, 1995; Abelmann and Elmore, 1998). Internally, the 
faculty and parents can examine school performance in an effort to improve its services. 
That is, internal accountability might be seen as a helpful diagnostic or formative 
evaluation that serves as a part of a school's continuous improvement effort. External 
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accountability often encompasses the oversight by government agencies, researchers or 
parents interested in making judgments about a program or school. These are generally 
summative assessments that inform decisions by policymakers about sanctions or 
rewards or edify researchers' understanding of school effectiveness. 

Results of the New Accountability measures described above generally provide more 
benefit to those outside schools than in them. That is, school personnel often see the 
results from such systems as being about them, rather than /or them. 

If schools are to improve on a large scale, I believe both internal and external 
accountability must be in place. Internal accountability plays a critical role in 
establishing the norms and culture of excellence within a school. It engages schools in a 
process of self-reflection about their own practice, a process schools have a natural self- 
interest in doing well. However, there are limits to self-referential evaluations, they lack 
objectivity (or the perception thereof) and thus may not provide the general public with 
an unbiased gauge of school performance relative to a state's standards. External 
accountability serves as an essential check on the system and provides schools with 
valuable feedback. However, it too needs to be informed by the internal expectations of 
schools. A system of schools is not likely to move toward increased performance if it 
ignores the expectations of the individual schools that comprise it. 

Abelmann and Elmore (1998) point out that there is a clear and critical tension between 
internal and external accountability. They state: 

The long-term fate of education reform as it is presently conceived lies largely in 
the tension between the uniform requirements of external accountability systems 
and the peculiarities of real schools. The new educational accountability systems 
will succeed or fail to the degree that they are designed with knowledge of how 
schools vary in their conception of accountability. 
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CREATING A COHERENT SYSTEM OF ACCOUNTABILITY 
In designing accountability systems, it is important to understand the components of 
effective schools and how government can serve to systematically enhance these school- 
level conditions. Building on the effective schools' research , Newmann et al (1997) used 
a four part rubric to measure school organizational capacity: 

• Teacher professional knowledge and skills 

• Effective leadership 

• Fiscal and curricular autonomy, and 

• Coherence of mission 

At issue is how can government create an oversight system that enhances and augments 
these components in all of its schools? 

A Working Theory of Action 

Figure 3A summarizes my working theory of how charter school accountability should 
work. The model is built on the premise that schools will improve from within more 
easily than they will from without (Newmann et al., 1997; Abelmann & Elmore, 1999). 
Therefore, this model suggests that government should design oversight systems that 
help build schools internal accountability in the hope that increased expectations will 
work to increase a school's organizational capacity and overall performance. 
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Figure 3A. Working Theory of Charter School Accountability. This figure 
summctrizes my working theory. It involves providing chctrter schools with the 
opportunity to meet local demands, central incentives to encourage performance, and 
capacity to support schools meet their goals. The arrow signifies the fact that the 
process is dynamic and that improvement requires continuous reassessment of goals. 




To understand how government could help schools build organizational capacity, I 
looked to successful private sector enterprises (e.g., Murnane & Levy, 1996)^ and 
promising public sector models. 
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Paul Hill and Robin Lake (1997) echo many of Murnane and Levy's (1996) principles in 
their plan for an accountability system to oversee the Washington State K-12 education 
system. They write: 

An accountability system must accomplish three tasks. It must attach 
consequences to performance so schools have strong incentives to improve 
instruction; create flexibility at the school level so that teachers and principals 
can improve instruction free of the impediments of unnecessary regulation; and 
ensure that schools get the help they need to improve, by encouraging formation 
of many different high quality assistance providers (p. 3).^ 

Drawing from these principles. Figure 3A includes three major components. One, 
incentives are meant to guide and catalyze schools and districts. Two, opportunity is 
meant to provide schools with the leeway to react to the states' external standards, and 
to give them the freedom to develop their own internal accountability. Three, capacity is 
needed to help schools improve the collective skills of their teachers and to improve 
their ability to use their resources in alignment with their mission. In theory, these 
components would work in concert to first improve the organizational capacity of 
individual schools, which in turn would lead to large-scale improvement. The curved 
arrow at the top of Figure 3A symbolizes the fact that the process is ongoing. As 
Murnane and Levy (1996) point out, successful businesses "persevere and learn from 
mistakes, there are no magic bullets" (p.l4). I will address each of these main 
components in turn. 

Incentives 

Accountability systems are often built around "rewards and penalties." These systems 
work on the principle that if government measures a school's performance relative to a 
common set of standards and then metes out penalties and rewards based on that 
performance, schools will respond by improving over time (Brooks, 2000).^ Standards 
and their accompanying assessments serve several purposes. First, they provide a 
common frame of reference. This addresses concerns that charter schools will create 
"balkanized communities" (Fuller, 2000, p. 254) in that common standards can serve to 
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weave all students into a common social fabric. Second, state standards and tests have 
internal benefit to schools as a helpful diagnostic or as reassurance that it is on the right 
track. Third, state tests provide a valid external check on performance, a check that 
taxpayers and lawmakers demand and deserve. 

However, standards and assessments have limitations. First, because cut scores on state 
tests are often designed to avoid intolerably high failure rates, they often serve as the 
minimum threshold of competence rather than a high bar of excellence.^ Therefore, tests 
will likely do little to inspire the performance of any but the lowest performing schools 
(Hoffman et al., in press). Second, such tests provide only a one-dimensional measure of 
performance.^ Therefore, if the Newmann et al. (1997) theory of internal accountability 
has merit, a one-size-fits-all assessment that does not take into account a charter school's 
own mission and site-specific goals will do little to encourage that school's ability to 
build its own organizational capacity. 

My assumption with regard to incentives in this model is that state accountability 
systems that take into account a school's mission and goals will have a greater positive 
effect on organizational capacity than authorizers that rely on a one-size-fits-all 
approach. 

An integral part of any incentive system is data. Internally, government can support 
charter schools if they can provide data that are timely and usable. Optimally, schools 
can use this evaluative data to design instruction and reallocate resources more 
effectively. Externally, parents and lawmakers can use evaluation data to make school 
choices and evaluate the public investment, respectively. The use of data at all levels is 
central to the continuous improvement of the system. 
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Capacity 

Another limitation of state incentive systems is that some schools may want to improve 
their performance, but may simply not have the human or fiscal capacity to do so. 
Anthony Alvarado, former superintendent of New York City's Community School 
District Two, explained that while incentives were important, they were not an end in 
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themselves. He said, "sticks don't inform, they just hit" (Hill et al., 2000, p. 161). He 
argued that more important than incentives was the need for capacity-building. He 
based his argument on his own limitations as a teacher when he said: 

The biggest problem in public education is that the adults don't know what to 
do... I thought I was a good teacher, [but] there were things I didn't know - what 
to do with certain kids at certain times. I knew I wasn't adding to their 
knowledge base. [I realized] 'it's not the kids, it's me' (Hill et al., 2000, p. 158). 

Despite the commonsense appeal of the rewards and penalty model, researchers argue 
that this model works on the flawed assumption that schools have untapped resources, 
i.e., that teachers are inherently lacking in motivation and that an incentive is needed to 
spur them to improve (Newmann et al., 1997; Massel, 1999). It is true that, some 
(perhaps many) teachers and school administrators are not working at their full 
potential, but many more want to improve and simply lack the capacity to do so.^ As 
Elmore (1996) points out, the problem with large-scale change is, "understanding the 
conditions under which people working in schools seek new knowledge and actively 
use it to change the fundamental processes of schooling" (p. 4). 

Capacity-building is important in any accountability system, but my assumption is that 
it may be especially important in the charter school community because the modal 
charter school has a less-experienced faculty and a leader with heavy administrative 
demands. In addition, while support can come from many private or public sources, in 
order for it to be of value to the school, this support should be sustained, on-site, and 
consistent with the school's mission (Darling-Hammond, 1993). 

Opportunity 

'The third component of the model is "opportunity." Elmore (1997) argues that 
accountability relationships should be based on reciprocity. He writes: 

Every demand for increased performance through a formal accountability 
system should carry an equal reciprocal obligation on the part of the party 
making the demand to provide the capacity to meet the demand. 
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In essence, authorizers' demands for increased performance should be matched with 
comparable school-level support and freedoms. If this does not occur, schools will have 
a justifiable argument as to why they could not meet the external standards. 

Charter proponents, such as the Center for Education Reform, define "strong charter 
laws" in terms of how much flexibility and independence a given charter law provides 
potential charter school founders. Hassel (1999) cites the following four provisions as 
important indicators of a strong charter school law: 

• no veto is possible by a local school board; 

• legal independence is possible;® 

• full per-pupil operating funds follow children; and, 

• minimal restrictions are put on the source and number of schools. 

My assumption is that opportunity is a double-edged sword. This definition of a strong 
charter school law, i.e., a law providing maximum opportunity, may yield relatively 
high numbers of schools and similarly high numbers of failures. Depending on an 
authorizer's lens, this can be seen as a positive or a negative. From a competition 
perspective, more charter schools means more market pressure and a greater chance that 
the system will improve. However, from a governance perspective, if rapid growth 
leads to the authorization of lower quality schools, a high level of opportunity would be 
seen as counterproductive. My analysis will examine the inherent tradeoffs in providing 
varying levels of opportunity. 

Limits of Government Oversight 

The primary intent of this working model is to improve academic performance on a 
large scale. It does not emphasize the importance of competition or school-level 
innovation, i.e., HasseTs (1999) competition, replacement, or laboratory theses. While 
innovation and competition may occur as a result of schools differentiating themselves, 
the focus of my model is on school improvement, period. The bottom line is that unless 
charter schools yield better performance, they will simply become another layer in the 
fossil record of school reform. 
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This said, there are two significant limitations to this charter school governance model. 
First, even if a working theory like this one serves to improve students' academic 
performance, it will not necessarily change their economic disposition. Fuller (2000) 
explains that fixing the schools is only part of a larger societal problem of inequality. He 
writes: 

Any policy leader who argues that simply reforming what goes on inside schools 
will make America a more fair society - without addressing child poverty and 
the deeper inequalities - is trying to fool you. To expect the learning curves of 
kids from working class and starkly poor backgrounds will rise substantially 
simply through better schooling, without attacking root causes, namely economic 
insecurity and constrained opportunities facing low income parents is hogwash. 
It ignores a half-century of empirical research, (p. 10)^ 

Second, charter schools and authorizers have significant limits on the discretion 
available to them. Since the U.S. Constitution delegates the power to oversee public 
education to the states, it might appear that the states have full discretion in defining the 
powers and responsibilities of charter schools.^^ However, federal civil rights laws 
supercede state law.^^ And as Jay Heubert (1997), attorney and professor of education at 
Teacher's College, Columbia University, points out: 

Federal disability law affects central aspects of charter school education -- 
curriculum, pedagogy, hiring, use of staff, school discipline, and budget 
priorities — at least as much as any other kind of government regulation, (pp. 
303-304) 

The Massachusetts and Texas charter schools in this study manage their own 
administrative responsibilities and are therefore deemed "local education agencies." 

This means they need to carry with them all the responsibilities of a district. Complying 
with all aspects of federal civil rights law means being willing and able to educate 
virtually any child that comes to their door. 
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Finally, the role of politics can not be understated. Several authors have noted the 
pivotal role and often unpredictable influence politics can have in shaping or derailing 
systemic reform efforts (Fullan, 1992; Elmore, 1997; and Hill et al, 2001). In the states 
studied herein, the competing interests within business, labor, and the legislature 
controlled the growth and autonomy of their respective charter school founders. 



The next chapter addresses my first research question; What are the organizational 
capacity needs of charter schools? If the intent of an accountability system is to improve the 
performance of the system, then the first step is to understand the problems the system 
is meant to fix. The subsequent chapters will explore how or if the working model 
described in this chapter of opportunities, capacity and incentive address these 
challenges. In reviewing the Massachusetts and Texas oversight models the reader 
should be asking, what is the incentive structure? Will these incentives encourage the 
development of organizational capacity and internal accountability? Do schools have 
reasonable access to capacity-building resources? Do schools in each system have the 
opportunity necessary to excel? 



Endnotes 



1 All 50 states now have some form of state testing program although Iowa does not have state 
standards (Geortz, 2001). Eighteen of them are considered "high stakes" accountability systems, 
i.e., states that tie significant consequences to poor test performance, e.g., state receivership or 
reconstitution (Education Counts, 2000). Massachusetts and Texas are two of the fourteen states 
that have high-stakes testing and charter school laws (Education Coimts, 2000 and USDOE, 2000) 

2 Richard Mumane and Frank Levy (1996) offer five common principles from "best practice 
firms:" 

1) Ensure that all frontline workers understand the problem. 

2) Design jobs so that all frontline workers have both incentives and opportunities to 
contribute to solutions. 

3) Provide all frontline workers with the training needed to pursue solutions effectively. 

4) Measure progress on a regular basis. 

5) Persevere and learn from mistakes; there are no magic bullets (14). 

^ Hill et al. (2000) analyzed seven major cities (Chicago, New York, Seattle, San Francisco, 
Memphis, San Antonio, and Boston) through this analytic frame of "opportunity, incentive, and 
capacity." 

4 The rewards for high performance might range from a cash allowance to increased flexibility. 
The penalties or sanctions for low performance could range from the bad publicity associated 
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with the public reporting of scores to complete dissolution of the school in the form of 
reconstitution. 

5 For example, the passing score on the Massachusetts Comprehensive Assessment System, 
MCAS, is a scaled score of 211, one point above failing. 

^ Relying solely on state tests as the sole measure of charter school performance is problematic for 
several reasons. One, charter schools are generally small (average enrollment of 200) and can 
have high turnover rates (up to 55% in some Texas charter schools). These two factors create test 
data that are highly variable. Two, charter schools are young, the average charter school is three 
years old (USDOE, 2000), so poor initial test scores may reflect developmental challenges rather 
than core problems with teaching and learning. 

^ Cremin (1961) (as cited in Elmore, 1996) points out that reformers often have unrealistic 
expectations of what teachers can do. Cremin writes: "From the beginning progressivism cast the 
teacher in an almost impossible role: [she] was to be and artist of consummate skill, properly 
knowledgeable in her field, meticulously trained in the science of pedagogy and thoroughly 
imbued with a burning zeal for social improvement. . .the gap between the real and the ideal was 
appalling" (p. 168). 

® "Independence" in this instance is defined as legal independence from the district. Because 
charter schools manage the majority of the administrative duties of the school, they are 
considered the "local education agencies" (LEA), and are responsible for the reporting as a 
district, e.g., special education services. 

^ See Coleman J.S. (1972) and Jencks and Phillips (1998). 

The state constitutions in Texas and Massachusetts delegate the primary responsibility of 
public education to their respective state legislatures, i.e., Texas Constitution, Article 1, section 7 
and Massachusetts Constitution, Part I, Article III, respectively. 

Federal laws relating to special needs students are Title II of the Americans with Disabilities 
Act, Section 504 and the Individuals with Disabilities Act (IDEA). See Heubert (1997) for legal 
review of charter schools and special education compliance. 
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Chapter 4: Autonomy: Understanding The Organizational Needs of 
Charter Schools 



School reform efforts often focus on the external levers necessary to push schools 
improve, but they often ignore how schools actually work (Tyack & Cuban, 1997). This 
chapter examines the needs of charter schools in an effort to highlight the areas where 
authorizers should focus their efforts. 

IMPROVING SCHOOLS FROM WITHIN 

To address the problem of improving charter school performance, one must first 
understand that government accountability is just one of the many demands on charter 
schools, and perhaps not even the most important one (Abelman & Elmore, 1998). 
Wohlstetter et al. (1995) and Hill et al. (1999) describe the complex web of accountability 
relationships confronting charter school leaders as "multidimensional accountability." 
Unlike public schools within a district, charter schools, as semi-autonomous entities, 
have the authority and need to negotiate directly with vendors, teachers, parents, 
funders, their governing board, their authorizer, and other governmental entities. 

In addition to this multi-faceted set of expectations, based on the argument described in 
the previous chapter, it is reasonable to hypothesize that 1) schools are more likely to 
improve from within rather than without;^ and 2) that faculty must use the building 
blocks of organizational capacity (teacher professional knowledge and skills, effective 
leadership, as well as fiscal and curricular autonomy) to cohere around a common 
mission and a set of school-developed expectations in order to produce a high-quality 
educational experience (Abelmann & Elmore, 1998; Bryk & Lee, 1993; and Newmann, 
Smith, Allensworth, and Bryk, 2001). In short, Newmann, Rigdon, & King (1997) 
theorize that schools with high levels of organizational capacity and internal 
accountability should produce strong academic gains. 
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If Newmann et al/s (1997) theory is correct, common state standards will have little 
impact on classroom experiences urJess those standards are consistent at least in part 
with the unique expectations of each school. In fact, their data suggest that strong 
external systems might actually "mitigate the development of organizational capacity" 
(p. 54) if a school's internal expectations are in conflict with a state's external standards. 

Given these findings, what if anything can government do to encourage the creation of 
high-performing school systems? Newmann et al. (1997) suggest that state agencies can 
do the following: 

• build organizational capacity through provision of technical resources (including 
aiding in the formulation of high standards for curriculum and student 
performance), 

• provide professional development to enhance teacher knowledge and skill, and 

• deregulate schools to strengthen school autonomy (p. 48; also see Newmann & 
Wehlage, 1995). 

Autonomy is a double-edged sword. My data revealed that charter school leaders in 
these two states felt significant fiscal and curricular autonomy and that the faculty in 
those schools generally had a strong understanding of their school's mission. However, 
my data also clearly indicate that with increased independence comes a significant new 
set of responsibilities. Further, the data indicate that in some cases, these added 
responsibilities impinge upon the core functions of schools: teaching and learning. The 
literature has generally focused on the problems related to start-up funding and facility 
financing. My research suggests that there are at least three other significant challenges 
to the organizational capacity of charter schools: heavy administrative costs; poor 
training and support for teachers; and limited leadership and governance capacity. 

THE NATIONAL PERSPECTIVE ON CHARTER CHALLENGES 

The National Fourth Year Study of Charter Schools (RPP, 2000) reported data from a 

survey in which they asked schools to rate the difficulty of each of 30 barriers from 1 to 
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5, with 1 being not difficult at all and 5 being very difficult. In reanalyzing this data, 
Mary Beth Celio folded these 30 barriers into the seven categories above (start-up, 
teachers, families, internal operations, regulations, district, and unions). The percentages 
in Table 4A represent the percentage of schools that rated these categories difficult or 
very difficult. 
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Figure 4A. Barriers to Implementation. This figure provides a summary of the 7 major 
obstacles to implementation reported by United States' charter school leaders in 1998. 




Source: USDOE, 2000; Celio, 2000. 

Charter school leaders ranked the following issues as challenges to implementation: 

• Start-up (50%) - inadequate financing, lack of pleinning time, locating facilities, and 
community opposition. 

• Teachers (43%) - difficulty in hiring and retaining teachers. 

• Families (38%) - lack of parental support, poor communication, difficulties in 
recruiting students. 

• Internal operations (30%) - weak internal processes, poor communication among 
faculty and administration, high administrator turnover. 

• Regulations (26%) - time-consuming accountability requirements, federal and state 
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regulations. 

• District (26%) - district/ central office resistance or conflict. 

• Unions (8%) - collective bargaining agreements, union resistance. 

These data reflect the findings of several researchers on this topic, namely, that 
"buildings and bucks" are the primary concern for these fledgling start-ups (Loveless, 
1996; Finn et al., 2000). 

Issue 1: Buildings and Bucks 

The importance of the facilities and financing problems can not be overstated. This 
basic difference between charter and district schools colors every decision a charter 
school board makes. If a charter school board can manage to get a lender to look past a 
school's limited collateral and the short time frame for loan amortization (usually five 
years), buying a facility puts the school in tremendous debt. That pressure to repay 
loans can influences financial decisions related to the selection of faculty and delivery of 
services. 

The alternative, renting a facility, not only limits the physical space in which a school 
works, it leaves the board vulnerable to unscrupulous landlords willing to raise rents in 
tight housing markets. The daunting cost of renovating or building a new facility has 
caused most charter schools to rent rather than build, e.g., in Massachusetts 37 of the 40 
charter schools in operation in 2000-01 (93%) were in rented or leased spaces. 

Issue 2; Administrative Burden 

The infrastructure required to run an independent public school is complex. Parent 
engagement (and student recruitment), internal operations, and regulations came up as 
the third, fourth, and fifth biggest barriers to implementation in the national charter 
school survey (Figure 4A). All of these functions could be folded into administration. 
Meeting the compliance requirements associated with civil rights laws was a critical 
challenge for many schools. The costs of administration in a Massachusetts charter 
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school are almost three times the percentage of expenditures allocated to a district 
school (21% versus 8%). This encumbrance is not often cited in the literature, but it is an 
important part of the charter school tradeoff; with the increased autonomy of charter 
status comes an increased administrative burden. 

This national survey data (Figure 4 A) is helpful, but it has limitations. Namely, it a) 
only provides the insights of the school leader, and b) represents a single snap shot in 
time. Because schools are complex, evolving organizations with many stakeholders, 
case studies of individual schools provide a richer understanding of the issues charter 
schools face. 

A CLOSER LOOK AT SIX SCHOOLS 

I examined three charter schools in Massachusetts and three in Texas. (A profile of each 
of the schools in my sample and detailed description of the selection process is provided 
in Attachment 2.). These schools were selected based on their relative performance on 
state tests. Recall that for the purposes of this study, the "exemplary" charter school in 
Texas was labeled "TX-A," the "acceptable" school was labeled "TX-B," and the "low- 
performing" school was labeled "TX-C." The three Massachusetts schools were 
similarly labeled - "MA-A", "MA-B", and "MA-C". 

A review of the demographic data illustrates that the six schools are generally consistent 
with the state and national trends described above (see Attachment 2 for details). On 
average, the schools: 

• are small (even MA-B, which serves 688, is still relatively small in that, on average, it 
only has 58 students per grade); and 

• are serving, on average, a higher percentages of African-American, Latino, and low- 
income students and lower percentages of special needs and bilingual students. 

However, the demographic composition of some of these schools differs considerably 
from their district averages. For example, while the average African-American student 
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population was 33% in the district, MA-C had an African-American population 
approaching three times that percentage, i.e., 89%. Similarly, TX-B had a Latino 
population that was nearly double that of the state average, 96% versus 54% in the 
district. The inverse was evident at TX-A. While the district average for white students 
was 44%, TX-A had a student population that was 80% white. These differences in 
demographics among charter schools and their district averages can have important 
implications for measuring performance. 

Understanding the School Labels 

The designations of the school, e.g., MA-A versus MA-C, indicate how each of these 
schools did relative to the state average on the MCAS in Massachusetts or the TAAS in 
Texas. That is, MA-A and TX-A have the highest average performance of the three 
Massachusetts and Texas charter schools in my sample, respectively. However, it is not 
clear that MA-A is necessarily performing better than MA-C. Coleman and others (1972) 
(also see Jencks, 1997) have demonstrated for thirty years that test scores generally 
reflect the students' backgrounds rather than how well a school serves children. 

Figure 4C, a graph of the percentage of low-income students across the three 
Massachusetts schools relative to their average eighth grade MCAS scale scores in 1999- 
2000, reflects this research. The graph shows that the higher performing MA-A also had 
virtually no low-income students. Conversely, MA-C, with 68% of students on free and 
reduced lunch, performed substantially lower on the MCAS. Thus, the demographics 
and accompanying MCAS scores of my three Massachusetts schools serve to reinforce 
the finding that there is an inverse relationship between poverty and performance. 
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Figure 4C. MCAS Scores vs. Percent Low Income. This graph summarizes the scaled 
scores of the three Massachusetts charter schools in my sample on the eighth grade 
English Language Arts exam (spring 2000) (top line) relative to the percentage of low 
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income students in the schools. 



Interestingly, while the data across the state of Texas follows this general pattern (see 
Just For the Kids website, Just4Kids.org), the three schools in my Texas sample do not. 
While TX-A, with no low-income students and the highest performance (98% of the 
students passing the eighth grade literacy portion of the TAAS) fits the pattern, TX-B 
and TX-C, are atypical (Figure 4D). TX-B, in an urban barrio and serving a population of 
students that is 90% low income, had a higher passing rate than TX-C with nearly a third 
the number of low-income students (29%). The analysis of organizational capacity that 
follows helps to explain why TX-B may have had a higher than expected passing rate 
and why TX-C appears to have struggled. 



51 

Herdman 



Figure 4D. TAAS Scores vs. Percent Low Income. This graph summarizes the percent 
"proficient" or "advanced" of the three Texas charter schools in my sample on the 
eighth grade English Language Arts exam (spring 2000) (top line) relative to the 
percentage of low income students in the schools. 




While these designations may not reflect how well a given school serves students, these 
designations do serve as triggers in state accountability systems and as such, represent 
important conditions to examine. That is, higher performing schools, like TX-A, will be 
eligible for positive incentives such as additional funding, while low-performing 
schools, such as TX-C, will be operating under the threat of sanctions such as the loss of 
its charter. Understanding how these designations influenced school efforts to improve 
is important to the analysis that follows. 

Measuring Organizational Capacity 

The measure of organizational capacity used in this study builds on the tools developed 
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by Newmann, King, and Rigdon (1997). The rubric is explained in more detail in 
Attachment 3, but the basic components are 1) teacher's professional knowledge and 
skills; 2) effective leadership; 3) fiscal and curricular autonomy; and 4) coherence of 
mission. Based on the rubric in Attachment 3, each school was scored on a scale of 1 to 
3, 1 being the lowest and 3 being the highest. 

Table 4C. Organizational Capacity of Six Charter Schools* This table summarizes the 
organizational capacity of the six schools in my sample. The ratings are based on the 
rubric explained in Attachment 3. 



Organizational 

Capacity 


MA-A 


MA-B 


MA-C 


TX-A 


TX-B 


TX-C 


Average 


Teachers’ professional 


2 


1.5 


1.5 


1.5 


1.5 


1.5 


1.6 


knowledge and skills 
Effective leadership 


1.5 


1 


1.5 


3 


2 


1 


1.7 


Fiscal and curricular 


3 


2 


3 


3 


3 


3 


2.8 


autonomy 

Coherence of mission 


3 


3 


3 


3 


3 


1 


2.7 


Composite Score 


9.5 


6.5 


9 


10.5 


9.5 


6.5 


8.7 



This analysis suggests that the charter schools in my sample had reasonably high levels 
of fiscal and curricular autonomy (average of 2.8) and with the marked exception of TX- 
C, coherent missions (2.7). 

In contrast, these schools had limited teacher professional skills and knowledge and 
inconsistent leadership as defined in the rubric. These two issues surfaced in the 
national survey data above (43% of charter school leaders reported difficulty in hiring 
and retaining teachers and 30% reported problems with weak internal processes, poor 
communication among faculty and administration, high administrator turnover) and 
these issues became even clearer in the case studies. 
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Issue 3: Teacher Training and Retention is Poor 

Table 4C indicates that the average score in relation to teacher's professional knowledge 
and skills was 1.6 out of a possible 3. Five of the six schools (MA-B, MA-C, TX-A, TX-B, 
and TX-C) scored 1.5, while MA-A scored 2. 

Charter Schools Were Able to Attract Talent 

MA-A teachers had the highest score in this area because, while their faculty was 
generally young, they had several veteran teachers and their content knowledge was 
strong. The Massachusetts Department of Education's site visit team notes: 

Broadly educated and talented teachers are devoted to the personal and 
intellectual development of their students.... MA-A students profit immensely 
from close contact with and coaching by a young, energetic faculty who hold 
multiple degrees from highly prestigious liberal arts colleges and universities. 

An additional cohort of. . .interns, mostly from Harvard, reduce already low 
teacher-to-student ratios even further. 

High Teacher Turnover Rates 

However, MA-A did not warrant a score of 3 for two reasons. First, there was a high 
turnover rate among the staff. The renewal visit report states: 

A unique curriculum, constant student coaching and democratic decision- 
making at the school exhausts even the heartiest. Only a few founding faculty 
members remain at [the school], and one reliable estimate places end-of-year 
faculty attrition in the Math Science and Technology domain at nearly fifty 
percent, (emphasis added) 

High turnover rates erode a school's ability to build internal accountability, i.e., a shared 
set of norms and expectations. A MA-A teacher highlights how the many demands of 
an independent public school can be overwhelming and how the impact of teacher 
turnover affects the organization. He says: 
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It's really hard. . .it's easier as a young single person. . .especially for the first 
couple of years, [this] was my life. So, now I'm pulling away - I don't have to 
spend as much time - I'm finding a balance. It's hard because a lot of people 
have come here over the years and then left. The turnover is not good for the 
institution and it's not good for the people who stay here because for example, in 
my group of six, we had three new people this year. Just to create a new group 
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Uncoordinated Professional Development 

The second reason MA-A received a score of 2 rather than a 3 was because its 
professional development efforts were generally unfocused. The faculty did receive a 
month's paid professional development time during the summer to develop an 
interdisciplinary curriculum, and the school is constantly redesigning its curriculum, 
which can be seen as professional development. However, the institution spent less 
than 1% of its budget on professional development and my interviews suggested that 
the use of those dollars was not well-aligned with the school's mission. A comment by a 
MA-A teacher reflects how most teachers responded to the question, "How are you 
supported as teachers?" 

[MA-A] is very open and supportive. . .but it's still up to us to initiate it or to do 

the application. I just need to take more initiative probably. 

The other five schools followed a similar pattern. The schools often attracted well- 
educated and passionate educators, but the conditions at the schools were often not 
conducive to their long-term development. The five other schools had teachers with 
limited experience (about five years or less) and high turnover rates. Certainly, young 
teachers can be as or more effective than more veteran teachers, often bringing energy 
and creativity to their schools. Independent schools have been thriving for decades on 
younger teachers willing to put their hearts and souls into their work. However, this 
high-energy output without sufficient support results in teacher burnout and turnover. 
The three Texas schools in my sample had turnover rates ranging from 22% to 56%. 
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Given the limited classroom experience of many charter school teachers, one might 
have expected professional development to be a priority. However, my data 
suggests otherwise. On average, the charter schools in my sample spent about 1 to 
2% of their budgets on professional development (which was consistent with state 
averages for charter schools: 2% in Massachusetts in 1995-96 (Herdman and Millot, 

2000) and 2% in Texas in 1998-99 (TEA, 2000, Part Two)). 

More important than the size of the allocation is how these dollars were utilized. My 
observations suggest that most of the schools used their resources in the same 
"drive-by workshop" approach that district schools traditionally do. Five of the six 
schools (all but TX-C) did dedicate a significant amount of professional development 
time to summer training focused on their school's educational model; however, 
throughout the year, training was generally sporadic and uncoordinated. For 
example, a second-year teacher from TX-A described his professional development 
opportunities as follows: "As far as developing, we are open and entitled to 
whatever the regional service center has - just like any other school. I don't feel like 
Tm missing anything." There was no indication in any of these schools, perhaps 
with the exception of MA-B, which was tied into the EMO's training plan, that they 
were using their autonomy to align their resources and training toward a common 
mission. 

There at least two economic factors that help explain why charter schools often hire less- 
experienced teachers and do not invest a significant amount into professional 
development. First, because these schools have so many start-up costs, governing 
boards have an incentive to hire less experienced and thus less costly faculty. Second, 
because the competing costs are high (e.g., facilities), professional development becomes 
deprioritized. 

These observations about teachers and their training highlight a promise and a challenge 
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of charter schools. The promise is that the teachers I met were generally well-educated 
and passionate, often drawn to the idea of starting something new. The same second- 
year teacher from TX-A quoted above said of his heavy workload, 

A lot of people in education know it is a. . .battle zone. Not everything is going to 
be handed to you. . . .It is really something you have to attack and take on and be 
passionate about.... I trust what I do, I trust the goal of this school and that 
motivates me a lot. 

However, the challenge is to work within the existing financial constraints to developing 
systems that support and retain these motivated teachers. The pattern of teacher 
burnout, combined with insufficient professional development, across all six schools 
does not bode well in the long term for building the organizational capacity these 
institutions will need to be consistently high-performing schools. 

Issue 4: Leadership and Governance Are Inconsistent 

The average score was 1.7 on the effective leadership criterion. One school (TX-A) had a 
score of 3, three schools had a score of 2 (MA-A, MA-C, and TX-B), and two schools had 
a score of 1 (MA-B and TX-C), Table 4C. 

A Strong Board and a Broad Skill Set 

TX-A had the strongest leadership structure. The board was well balanced (with legal, 
business, and education expertise), and the administrative team consisted of leaders 
with both business and educational expertise. The school director is an attorney with 
management experience, and she has hired educational leaders for the upper and lower 
schools. She explained her history this way: 

I was one of the founders of the school and on the board for the last five years. 

This year — because the challenges we had were not. . .curriculum-driven — the 
board asked me to come in and help. We had a lot of educators interested in 
curriculum and teachers with a lot of passion and talent, but we didn't have 
human resource capabilities and definitely lacked facilities and finance 
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infrastructure. 



This principal was able to set up a human resource plan and secure bond financing for a 
new facility .2 Her careful attention to the finances of the school will certainly have short- 
and long-term benefits for the TX-A students. For example, at the moment, TX-A is able 
to dedicate 69% of its budget to the classroom, as compared to TX-B (57%), TX-C (27%), 
and the state (58%). Long-term facility financing will free up resources each year that 
can also be plowed back into the school, rather than being lost in interest or lease 
payments. 

The Right Leader at the Right Time 

TX-B's performance may have benefited from strong leadership as well. Teachers 
described the first few years of the school as "chaotic." The one remaining teacher from 
the original cohort said: 

The initial leader of the school meant well, but she wasn't organized, we [the 
teachers] were all on our own. Everyone was doing something different; we 
couldn't get supplies. . .it was a mess. [Maria] came in two years ago and things 
have stabilized. Teachers are staying longer because we know what is expected 
and we get help when we need it. She gives all the teachers the goals for the 
week by grade level - we helped establish these goals - but she makes sure we 
are in sync on each grade level so there aren't any gaps. 

The TX-B leader had a different skill set than the one in TX-A, but she appears to have 
been the right person for the needs of that school. 

The Leadership Void 

In contrast, TX-C went bankrupt in the fall of 2000 largely due to its poor leadership. 

The school grossly mismanaged its resources and had no clear oversight at the board 
level. TX-C was founded by one of the pioneers of the charter school movement in the 
state. Urtfortunately, the governing board did not provide the school with adequate 
fiscal guidance. This was due in large part to the fact that the board had no 
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administrative distance from the school's day-to-day operations. The founder's wife sat 
on the board and the founder himself was the board chair Even prior to the school's 
closure for being $1.5 million in debt, this poor leadership clearly had a negative impact 
on the school's learning environment. One example is in how resources were allocated. 
The ratio of classroom to administration expenses was 1:1 in TX-C (as compared to 7:1 in 
TX-A). The moral in the TX-C story is that governing boards play a critical role. 

Alternative Solutions to the Problem of Inconsistent Leadership 

Charter schools are often seen as leader-dependent organizations because they are 
independent from local governments. However, my data revealed that some schools 
performed at relatively high levels despite poor or inconsistent leadership. For example, 
MA-A and MA-B both have had five principals in as many years, yet MA-A had some of 
the highest MCAS scores in the state and MA-B was recognized as an exemplar school 
by the state for its MCAS gains in 2000. MA-A scores could reasonably be dismissed by 
the fact that the students would have done well in any setting because 94% were white 
and less than 1% were low income (Attachment 2). However, my observations suggest, 
and the state's site visits support the fact, that there is a clear ethos of excellence at the 
school, a consistent internal accountability. This can be attributed to a steady and 
thoughtful board and a commonly understood set of pedagogical principles. Despite 
annual turnover at the school, the teachers that worked there knew what the guiding 
principles of the school were before they arrived. In short, at this school, effective 
leadership developed collectively rather than being embodied in a single person or 
executive team. 

In MA-B, the leader has turned over annually for the last five years and the board has 
been cited in state site-visit reports as being "micro-managing." In this case, the school 
appears to have been able to maintain a sense of academic consistency because of the 
structure and support provided by the EMO. For example, when the school was in 
transition between principals or trying to train a new principal, the EMO hired the 
principle of a second, more successful, charter school in Massachusetts to serve as a 
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principal mentor or coach. This said, the lack of effective leadership in the day-to-day 
management of the school had a direct bearing on the faculty and students. One MA-B 
teacher said: 

It was huge. I mean if [the EMO] did not act on the last director, I would not 
have returned next year. . ..This school was in chaos, it was in crisis 
management... it was a complete meltdown.... There was no follow 
through.... Even if you went and did the work and did the memo and did the 
meetings and everything else and were met with "Absolutely!" — that could be 
the last you heard of it. And the kids felt it because it was a very agitated 
atmosphere, it was a very bipolar kind of operation.... Being in a banquet hall for 
19 weeks is pretty egregious....! said, "I cannot teach here. Tm doing the best 
that I can— my kids are not learning here." It was a ridiculous expectation. So, 
did it affect them? Profoundly, every day. 

Governing an independent public school is a formidable task. With more opportunity 
comes more responsibility and greater need for capacity building. All six of the schools 
had turnover at the principal level and most had problems clarifying the roles and 
responsibilities of their governing boards. In the first four years of operation, the 
number of principals at schools in my sample ranged from a high of five (MA-A and 
MA-B) to a low of two (MA-C, TX-A, and TX-B). With the exception of TX-A and MA-A, 
the other four governing boards appeared to have problems in clarifying their role[s] 
and/ or finding members that had the requisite experience and competence to manage 
these schools. 

These findings point to four larger issues. First, the data indicate that finding governing 
boards with the competence and skills to manage a large number of public schools is a 
challenge. Second, the skill set required to run these schools differs from a traditional 
school and may require leadership from a team rather than an individual (e.g., TX-A). 
Third, that leadership can come in many forms, e.g., while MA-A relied on the steady 
vision of its board, MA-B used the help of its private partner. Fourth, the lack of 
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leadership in some form can be disastrous, e.g., TX-C. 

Newmann et al (1997) hypothesized that schools would be more likely to build internal 
accountability, i.e., norms of excellence, if they had fiscal and curricular autonomy; 
mission-driven organizations; teachers with high levels of professional skills and 
knowledge; and effective leadership. In fact, Newmann et al (1997) state explicitly that 
charter-like schools, based on a mission and with significant local control, would be 
more likely to build internal accountability. However, because these assumptions were 
built on research done in restructuring schools within traditional school districts, the 
researchers may not have foreseen the challenges associated with creating schools that 
have significant fiscal and curricular control. 

Autonomy Brings Unanticipated Challenges 

The charter schools in my sample were autonomous and mission-driven, the first two 
components of the Newmann et al criteria. (The average scores of 2.8 and 2.7 out of 3 on 
my rubric, respectively, support this observation.) However, with that independence 
came unanticipated challenges. Because these schools were independent from their 
respective districts, they had to shoulder extensive start-up, facilities, and administrative 
cost as well as manage the complex governance issues associated with running a public 
institution of choice (e.g., developing a board, serving all students, and meeting the 
needs of parents). These financial and governance challenges may help explain why 
leadership effectiveness and teacher skills were relatively low in my sample (average 
scores of 1.7 and 1.6, respectively). That is, the demands that come with being an 
independent public school may serve to make the leadership challenge even more 
difficult and they may serve to deprioritize the allocation of resources to the core 
enterprise of schooling: classroom instruction. 

An important related point is that a school's capacity is not a function of time. As my 
case studies demonstrated, all six schools varied considerable in terms of their capacity 
despite being operational for roughly the same amount of time. Having autonomy may 
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enable charter schools to develop internal accountability, but it does not ensure it. 
Policymakers should understand this variance and develop strategies to address it. 

Charter schools represent a tipping of the scales of governance from central toward local 
control. However, with this shift, there needs to be an understanding about what 
elements of schooling should be controlled centrally and what should be controlled at 
the school level or as Elmore (1993) states, "what is loose and what is tight?" 

The intent of this chapter was to illuminate how charter schools work as organizations in 
an effort to inform how authorizers can use their limited resources to improve these 
schools. The challenge is how government can work to facilitate charter school growth 
in these areas. The next chapter pulls the lens back and examines the actions of the 
authorizers. It examines the key policy decisions that have affected the trajectory of 
their respective charter school initiatives. In so doing, I return to the analytic frame of 
opportunity, capacity, and incentives to examine those decisions and their associated 
tradeoffs. 



Endnotes 



* See the following for a discussion of building capacity: Spillane & Thompson (1997); Darling- 
Hammond, 1993; and Century, 1999. 

2 TX-A was the first charter school in Texas to establish a credit rating and to secure bonds. 

^ This weak board construction was likely a remnant of the original law which said that the CEO 
was the grantee. The legislature's emphasis on improving the board structure did not come until 
two-three years after TX-C was operational. 
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Chapter 5: Opportunity and Capacity 



This chapter examines the opportunity or flexibility available to charter schools and the 
capacity authorizers provide in helping schools meet the challenges raised in the 
previous chapter. 

The major findings in regard to the opportunity and capacity of the schools in my 
sample were: 

1) Charter schools had significant curricular and fiscal autonomy, but feared that those 
freedoms were becoming re-regulated over time. 

2) Autonomy may be an enabling condition for success, but it does not ensure it. 

3) Authorizers have limited capacity to help address charter school challenges. 

4) A broad set of public and private partners is essential to the support of charter 
school initiatives. 

OPPORTUNITY: CORNERSTONE OR CHIMERA? 

Opportunity as Cornerstone. Nationally, most charter schools report having high levels of 
control over a wide array of school operations. The National Charter School Study, 
Fourth Year Report (2000), found that charter school leaders had primary control over 
their administrative operations, including budget (73%), purchasing (88%), and hiring of 
teachers (88%). Charter school leaders also reported generally high levels of control 
over their education program, including scheduling (95%), curriculum (83%), and 
student assessment (72%). 

Charter leaders in both Massachusetts and Texas charter schools reported high levels of 
autonomy (Celio, 2000). My case studies tended to support this finding (the average 
across the six schools was 2.8 out of 3). Moreover, the stakeholders at these schools 
consistently stated that the freedom afforded to them as a charter school — e.g., the 
ability to make fiscal and human resource decisions about the design of their school — 
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was the most powerful part of their charter status. 



What did they do with this freedom? Some used it to hire mid-career professionals who 
were very skilled but unwilling to go through the state's teacher certification process, 
e.g., a professional writer at MA-A or a former electrical engineer at TX-C. Some, like 
MA-B, used their freedom to bring the international experience of an educational 
management organization (EMO) to the children in their community. Others, such as 
TX-B and MA-C, used this opportunity to create a school around specific populations of 
students that were not being well-served in their communities, e.g., low-income African 
Americans in the case of MA-C and Latino immigrants in the case of TX-B. Still others 
saw their charter as an opportunity to bring a new level of academic rigor to their 
communities, e.g., TX-A's focus on the International Baccalaureate. In short, the six 
schools in my sample saw autonomy as a critical component of the charter school 
experiment and a key difference between them and their district peers. 

Opportunity as Chimera. While autonomy is a cornerstone of the charter's "basic 
bargain," some found that there were more limitations on their flexibility than they 
anticipated. Compliance with federal disability law was perhaps the biggest hurdle. As 
the primary administrators of special education services at their schools, open- 
enrollment charter schools in Texas and Commonwealth charter schools in 
Massachusetts, are their own Local Education Agencies (LEA). This means that these 
schools have the benefit of controlling their own resources and the responsibility of 
meeting the same reporting requirements as a district. However, unlike districts that 
have had decades to build institutional knowledge on how to comply with federal civil 
liberties law, charter school leaders are often starting from scratch. The principal of TX- 
A described the steep learning curve and the heavy reporting demands this way: 

When we first got started they said we could design our own curriculum, do all 
these things that you didn't have to do in a public school and then "wham!" 
Special Ed is a major issue here and. . .we can't deviate from the norm on that. 

We can't come up with our own program as far as using the federal money. We 
need something that will help us through the maze of paperwork. The 
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Charter school leaders in both states argued that their authorizers were gradually 
drawing them back into a system of oversight by compliance rather than performance. 
The upper school principal of TX-C describes his concern about the increasing loss of 
control this way: 

What I've seen is that they want us to have freedom and uniqueness., yet more 
and more they're wanting us to comply with the commissioner of schools. That's 
what's happening if you want to know in a nutshell what's happening in Texas. 
We've been granted charters and uniqueness, but [it seems the TEA is saying,] 
"we're going to have to start keeping them in a box..." 

In Massachusetts, a founding board member from MA-A argues that the increasing 
importance of the Massachusetts Comprehensive Assessment System (MCAS) is 
working to erode the original freedoms established in their charter. She said: 

I think Massachusetts has a pretty good balance [between autonomy and 
accountability]. It's shifting all the time so I can't say that the balance today is 
the same as the balance yesterday. When we put in our charter there wasn't 
MCAS and the state framework. So that's not in our charter and then suddenly 
some regulatory committee says you have to [respond to this]... The more 
Massachusetts shifts in that direction, the less good the balance is. 

Finally, a parent in TX-A suggests that the lack of clarity in the regulations may be 
causing schools to "over comply."^ That is, charter leaders may be designing policies 
within tighter constraints than the ones they are required to meet. She said: 

If we are not going to be forced to live by these rules, then tell us that! But give 
us the rules. You can go anywhere between here and here. You're not really 
sure which step to take. A lot of times when you have much younger 
administrators who may not have had much experience with Austin, they tend 
to not want to step because they have seen their colleagues get their hands 
slapped so horribly that they are afraid to move and so therefore we stay still. 
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In sum, the flexibility provided to the charter school leaders in my sample enabled them 
to have the "fiscal and curricular autonomy" thatNewmann et al. (1997) argued was 
essential to building organizational capacity. However, my findings also suggested that 
this freedom was perhaps not as broad as charter school founders anticipated and may 
be narrowing over time. 

Based on my school-level case studies, this freedom appears to be a basic enabling 
condition. Without it, the schools could not align their resources around a common 
mission or respond to their community's needs. However, as TX-C exemplified with its 
uncoordinated use of resources and ultimate closure, school-level freedom does not 
automatically lead to increased organizational capacity. The critical issue in regard to 
opportunity and its influence on a school's organizational capacity appears to be in how 
that flexibility is used. School-level opportunity enables, but does not ensure, the 
improved use of resources and subsequent improvement in a school's organizational 
capacity. 



CAPACITY: MAXIMIZING PRIVATE AND PUBLIC SUPPORT 
In discussing charter school accountability, there are school-level and authorizer-level 
capacity issues. As discussed in the previous chapter, these semi-autonomous public 
schools face some of the same issues district schools confront (e.g., teacher training and 
inconsistent leadership) as well as a few that are unique to charter schools (e.g., facility 
financing and start-up funding). The authorizers, located in their respective state 
capitals of Boston, Massachusetts, and Austin, Texas, face the additional challenge of 
how to support these schools from afar with limited budgets. 

Authorizers and Their Limited Capacity 

Once the charter school laws were enacted in Massachusetts (1993) and Texas (1995), 
state authorizers were charged with interpreting those laws and implementing them as 
policies. They needed to develop regulations for how to select, monitor, and evaluate 
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charter schools. 



The Massachusetts and Texas departments of education are broken into clusters of 
cubicles based on different pots of funding or specialization, e.g., Title I, Special 
Education, or after-school funds. Over the decades, each unit has developed its own 
vocabulary as well as tight guidelines specific to their funding stream. In this culture 
that one authorizer described as "compliance land," authorizers are asked to do 
something that is difficult to quantify or codify - use their judgement. They must make 
subjective decisions about the potential of a charter applicant's proposed educational and 
business plans. Ultimately, authorizers are charged with deciding whether a school was 
"successful" during its five-year charter. In this risk-averse culture, the job of an 
authorizer is indeed enigmatic and uncharted territory. 

Authorizers are the proverbial "middle men." In both Massachusetts and Texas, the 
authorizers are part of their respective state agencies, the Massachusetts Department of 
Education (MDE) and the Texas Education Agency (TEA). As such, they are agents of 
their respective State Boards of Education and legally beholden to (and funded through) 
the laws established by the legislative and executive branches of government. 

In turn, authorizers oversee the charters they grant. They not only manage the 
distribution of state and federal funds to charter schools, but they are obliged to 
troubleshoot for them as well. It is the authorizers who field panicked calls from charter 
founders who have lost their facility just months before their students are set to arrive 
and will work with charter founders to help to clarify their tax status. 

The authorizers in Massachusetts and Texas had much in common when they first 
began. Both states' authorizers wrote their first charter school applications in the mid- 
1990s (Massachusetts in 1994 and Texas in 1995) and both states began with 25 or fewer 
available charters (Massachusetts had 25 and Texas 20). At that time, there was a single 
point person and the equivalent of one or two full-time employees who staffed the 
respective offices. In Massachusetts, Secretary of Education Piedad Robertson 
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designated Virginia Grieman chief counsel for the office, and assigned two half-time 
analysts to head up the drafting of the original charter regulations. In Texas, Brooks 
Flemister worked with one full-time assistant to get the Texas charter initiative off the 
ground. 

Because neither state dedicated significant additional funds to manage these initiatives, 
both offices were funded in large part by federal charter school funds. Also as a result of 
limited funds, both charter school offices (called Charter School Division in Texas) 
stayed relatively small through 2000. Each office had a director (or Associate 
Commissioner in Massachusetts) and two or three staff.^ However, in 2001, both state 
charter school offices have increased their staffing considerably. The Texas Charter 
School Division now consists of the director and nine other staff and the Massachusetts 
office has increased its staff to seven full-time members. It was through these small 
administrative offices that the initial charter school founders worked to create their 
''dream schools." 

Capacity-building Efforts 

These authorizers had a limited ability to help (and in the case of Massachusetts, interest 
in helping) all schools they had sponsored. Philosophically, Massachusetts tended to 
not want to provide direct technical assistance to the schools they chartered because that 
office determined that providing such assistance might muddy the waters when it came 
to hold these schools accountable for performance. That is, if school X is failing and yet 
it complied with its authorizer's training request, then the authorizer might be in a bind 
when it comes to closing that school for poor performance. Moreover, even if an 
authorizer wanted to provide technical assistance, it would be very challenging 
logistically to meet all of their schools' needs simply because the authorizers could be 
hundreds of miles away from a given school. Moreover, authorizers generally don't 
have the resources to address all of the needs of charter schools. To review, the charter 
issues highlighted in the previous chapter were: facilities, start-up finding, 
administrative burden, poor retention and training of teachers, and inconsistent 
leadership and governance. 
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Therefore, Massachusetts and Texas charter school leaders utilized a rich mix of public 
and private providers to support their efforts. Figure 5E summarizes six capacity- 
building vehicles in Massachusetts and Texas charter schools: 

1) Direct Authorizer Support 

Direct central support from authorizers ("1" on Figure 5E) encompasses state programs 
designed to support charter schools. Fiscal support can come in the form of state 
funding dedicated for charter schools (e.g., state tuition or a facility financing fund) and 
site visits by state agency personnel can provide technical support. 



Figure 5E. A Map of Capacity-Building Resources 
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*Note: Districts are not included in this figure because while they provide significant support to 
many charter schools, they did not provide a significant amount of support to the state-authorized 
charter schools in my sample. The primary use of district services was through individual 
contracts, e.g., special needs or food services. This type of interaction falls under independent 
contracting (4 in Figure 5E). 

Government has provided the primary financial support for charter schools (Table C, 
below). The state average cost per student in Massachusetts and Texas was the core 
funding source for most charter schools: 86% in Texas (UT, 2000) and 87% in 
Massachusetts (Herdman & Millot, 2000; MDOE, 2001). Further, as will be discussed in 
the next chapter, authorizers helped these schools identify problems via the site visit 
process. 



Table 5C. Summary of Charter School Revenues. This table summarizes the state, 
federcd, and other funding sources for the six schools in my sample. 



School 


State Tuition 
Amount Percent 


Government Grants 
Amount Percent 


Private 

Amount Percent 


MA-A 


2,290,152 


83% 


23,439 


>1% 


453,063 


16% 


MA-B 


5,368,298 


91% 


327,577 


6% 


224,775 


4% 


MA-C 


2,764,988 


89% 


18,100 


>1% 


167,342 


5% 


MA Charter Average 




87% 




9% 




3% 


TX-A 


2,065,000 


87% 


0 


0% 


300,000 


13% 


TX-B 


1,120,655 


91% 


98,520 


8% 


17,500 


1% 


TX-C 


4,013,056 


94% 


250,000 


6% 


10,000 


>1% 


TX Charter Average 




78% 




13.5% 




7% 



Sources: Massachusetts averages are from MDOE, 2001 and the school averages are from 
independent financial audits of individual Massachusetts charter schools in 1999/2000. 
The Texas data is from PEIMS (2000) on the Texas Education Agency website. 



2) Indirect Authorizer Support 

Indirect authorizer support ("2" on Figure 5E) can come in the form of funds provided 
through federal pass-through grants or services provided through regional government 
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offices (e.g., Texas' 20 regional educational service centers), see discussion below, 
"Taking a Closer Look at Regional Service Centers." 

The USDOE's charter school start-up fund has been instrumental in addressing a whole 
range of capacity needs.^ The federal government has provided a considerable 
supplement (up to 13.5% in Texas and 9% in Massachusetts, Table 5C). The Public 
Charter Schools Program (PCSP) was established in 1994 and has provided a total of at 
least $12 million to Massachusetts and $24 million to Texas. With few restrictions, some 
schools used the funds to improve their libraries or buy software, while some used the 
funds to simply make payroll. Some schools have also have received direct grants from 
the federal government, e.g., TX-C was the recipient of a $500,000 award for 
disseminating best practices. In short, the federal charter school program was very 
useful in that, unlike many state and federal grant programs, it provided large sums of 
funding with few restrictions and reporting requirements (Herdman & Millot, 2000). ^ 

3) Government Outsourcing 

Government outsourcing is another valuable way to extend the capacity of a state 
authorizer ("3" on Figure 5E). An example of this is Massachusetts' use of a private, 
nonprofit organization to conduct the state's renewal visits. 

Government outsourcing can provide excellent service but it can be expensive. The 
Massachusetts site visit contract enables the state not only to coordinate an oversight 
function that it simply does not have time to do, but it a) brings a collective skill set to 
the oversight process that the authorizer does not have, and b) it provides some political 
distance to the oversight findings. The obvious drawback to this contract is its cost. At 
roughly $19,000 per visit, providing renewal inspections for every school over several 
days for a growing number of schools becomes financially prohibitive. 

4) Independent Contracting and Resource Centers 

Independent contracting encompasses schools using their resources to purchase their 
own services from private vendors ("4" in Figure 5E). Newly created charter schools 
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purchase a range of back-office services from outside vendors (37%). In addition, a 
growing percentage of charter schools are managed by private management companies 
(such as MA-B), which provide services that range from administrative duties to 
educational and management functions (Hill et al., 1999).5 

Independent contracting gives the schools full control of their capacity-building; 
however, it may not take full advantage of the economies of scale associated with group 
purchasing. In addition, relying on the resource centers for technical assistance largely 
alleviates the tension of an authorizer that has the potentially conflicting responsibilities 
of both providing services and evaluating a given school. The disadvantage of relying 
solely on charter school resource centers for technical assistance is that, as privately 
funded organizations, these resource centers may not feel an obligation, or have the 
capacity, to provide services to all schools. Further, if the economic markets take a 
downturn, and private dollars become tighter, it is unclear how long these Centers can 
continue to provide their services free of charge. 

Charter school resource centers like those in Massachusetts and Texas also provide 
important services, often free of charge. Both states have charter school resource centers 
that provide a range of services, such as direct on-site training, information sessions for 
potential charter school applicants, coordinating conferences, and commissioning of 
research on charter-related topics. These resource centers are funded through private 
sources, so their services are generally free to charter school leaders. 

5) Charter-to-charter support 

Charter schools are also beginning to help themselves ("5" in Figure 5E). In 
Massachusetts, individual schools developed cooperative arrangements with one 
another where they shared a special education specialist. In 2001, a newly created 
Massachusetts charter school association is looking to formalize and expand on this 
concept. 

6) Private foundations 
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Private foundations have been very helpful in providing targeted resources to the 
charter schools in my sample ("6" in Figure 5E). For example, foundations such as the 
Annie E. Casey, Walton, and DeWitt Wallace have funded the development of school 
infrastructure and assisted in the charter school start-up process. Massachusetts and 
Texas charter school leaders have raised about 3% of the average charter school budget 
from private sources (MDOE, 2001; TEA, 2000, Part One, Table III.8). 



A Closer Look at Regional Service Centers 

Texas' regional service centers could play an important role in delivering technical 
assistance and professional development. (Massachusetts does not have an analog to 
the regional service centers.) Texas' 20 Regional Education Service Centers took some 
time to understand charter schools, but they appear to be making progress. In the 
spring of 2000, the three Texas charter school leaders in my sample commented that the 
state's charter school office and the regional service centers were helpful in providing 
information, but often were not quite sure how best to help. The director of TX-A 
summarized the support from the state and regional centers thus: 

I think they [TEA] need to have a support system in place even though the 
service centers try to do a good job. I think at times the [service centers] don't 
know what to do with this animal either — the charter school. 



73 

Herdman 



Figure 5F. A summary of services provided by a Texas Educational Service Center. 
This chart summarizes the distribution of training provided to charter schools in the 
Houston area. 



leadership 

12 % 



instruction 

13 % 




compliance 

75 % 



Source: Regional Service Center 4, Houston, Texas. 

When told of this comment a year later, a charter school specialist from the education 
service center in Houston was not surprised. However, he insisted that while it took a 



significant amount of service to charter schools. He said: 

Early on, we were just told that we would be working with [charter schools]. We 



now we have dedicated staff and do quite a bit of work with them. For example, 
I coordinate monthly meetings with all the charter school leaders in my region. 
They all don't come, but the ones that do seem to benefit. 

In fact, a review of technical assistance provided by the regional service center serving 
TX-B shows that the center has been quite active. From August 1, 2000, to February 28, 
2001, the service center had been in contact with the region's 46 charter schools nearly 



couple of years for the service centers to get up to speed, they were now providing a 



didn't have the staff for it and we didn't know what their needs were. However, 
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2,000 times (1,981). Most of those site visits or training sessions were focused on 
compliance training (75%, or 1,492) and more than a third were focused on the use of 
training of PEIMS, the Public Education Information Management System (805 sessions, 
or 41% of the total). Table 5F.6 However, 13% (249) of the training sessions were 
dedicated to core service issues such as reading and language arts, and 12% (240 
sessions) of the training was used to address infrastructure issues such as leadership 
development. More specifically, TX-B accessed this regional service center 96 times in 
2000 and bought 27 professional development days. 

Regional service centers could be valuable resources to the state and the charter schools. 
From the state perspective, they provide a vehicle for building charter school capacity 
across the state, something the Austin charter school office simply can not do in such a 
large state. In addition, because they are affiliated, but distinct from the TEA, they 
provide some distance between the capacity -building and accountability sides of state 
government. From the school perspective, if these Centers are providing high quality 
services, they have a low cost and local means of building their capacity. 



Discussion of Capacity-Building Options 

It is clear that the state authorizers do not have the fiscal or technical resources to meet 
these needs, therefore, how should government spend its limited resources? A range of 
private and public providers has filled the breach, but it is still unclear whether this 
patchwork of funding and services will be enough to help these schools thrive. Should 
government dedicate most of its limited resources to developing its incentive systems or 
does it have an obligation to provide technical assistance as well? If government does 
dedicate its resources to technical assistance, would schools see them as the best 
provider? What level of obligation does government have in providing charter schools 
with the financial resources necessary to contract out for these services, especially if 
government is competing for these contracts as well? 

Each capacity-building source comes with trade-offs. Private grants can be larger than 
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government grants with fewer reporting requirements. However, private funds, or soft 
money, like other grants, are usually given to schools for a short period of time, generally 
one to three years, and thus should not serve as the foundation of a school's operating 
budget. Private providers have more of a motivation to provide a higher quality 
training opportunity, but they are generally more expensive than the public sector 
offerings. Texas' regional service centers were highlighted because they offer a 
subsidized-private answer to the technical assistance question and are available 
regionally. Thus, they have the potential of being a low-cost option of a high quality 
supply of services. 



The next chapter examines the last leg of this three-part theory: the incentive structure. 
The focus will be on examining the costs and benefits associated with the quantitative 
and qualitative measures of student and school performance. 



Endnotes 



^ CPRE (1992) notes that "over-compliance" is common phenomenon. 

2 In Texas, there have been only two directors: Brooks Flemister (1995-2000) and Dr. Susan Barnes 
(2000-present). In Massachusetts there have been seven point people in seven years. The 
succession of personnel heading the charter school office from 1994 to 2001 is: Virginia Grieman, 
Lisa Blout, James Peyser, Paul Herdman, Scott Hamilton, Edward Kirby, and Susan Miller 
Barker. Ms. Barker vacated the position in 2000. 

3 The Public Charter Schools Program (PCSP) was established in 1994 to address the start-up 
costs of charter schools. As of 1998, the PCSP also allows federal support for older charter 
schools to disseminate information about their successful practices. The fiscal year 1999 
appropriations for PCSP is $100 million. Charter schools receive their subgrants from the state or 
in a few cases, directly from ED. 

^ As reported in Herdman and Millot (2000), many state grants are for under $500 dollars and 
their use is restricted to specific purposes. For most Massachusetts charter schools in 1996, the 
small grant amounts were not worth applying for given the associated restrictions and reporting 
requirements. 

5 Legg-Mason (1996) reported that educational management companies had captured about 10% 
of the charter school market in 1996 and that by 2010 that EMO's would control 10% of the entire 
K-12 market. 

^ For example, the director of TX-A reported her frustration at having to create a dual reporting 
system; one for the state and one for the school's internal benefit. She said: 

Just the chart of accounts is a real nightmare because to run a small business you have to 
have a different sort of accounting profile. I mean what we produce for them [TEA] 
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helps us almost not at all. So we either have to rim two totally separate groups of 
numbers or we have to be less informed about our own business because we are sifting 
through many, many, categories that have little relevance to what we do. 
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Chapter 6: Incentives 



Incentives are the signals that are set in oversight policies regarding what is important 
and what is not. Ultimately, those motivations are defined most clearly when sanctions 
or rewards are meted out for performance. 

This chapter examines the tradeoffs in developing an oversight system through the 
frame developed by Hassel and Herdman (2000): 

• Setting the terms - the process for selecting charters, agreeing on terms within those 
initial contracts, and developing accountability plans (i.e., long-range goal-setting 
documents). 

• Gathering the data - the processes for measuring school and student performance, 
limited to a discussion of test scores and on-site visits for this paper. 

• Using the data - the process of using the data collected to make a decision as to 
whether a charter should be renewed or not. 

In this chapter, I argue four points. First, that the goal setting processes are not as 
inclusive as policymakers might have envisioned. Second, that authorizers are generally 
focusing more on compliance than performance. Third, that tests score data and site 
visits serve critical complimentary roles. Fourth, and finally, that shutting a school 
down for academic performance is a daunting political and technical challenge. 

A Brief Summary of the Charter Laws in Massachusetts and Texas 
Massachusetts passed its charter school law in 1993 as part of the omnibus piece of 
legislation known as the Education Reform Act of 1993 (MERA). Texas followed in 1995 
with charter legislation serving as a centerpiece of the legislatures rewriting of the Texas 
Education Code. 

There are many commonalties in the legislative processes that led to the passage of 
charter school legislation in Massachusetts and Texas.i Both states had Democratically- 
dominated legislatures and Democratic governors a year prior to the passage of their 
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respective charter school statutes. Both states elected Republican governors the year 
before passing legislation (William Weld in Massachusetts and George W. Bush in 
Texas, respectively). The arguments made for the passage of these laws encompassed all 
of the theses discussed in earlier chapters, i.e., laboratory, competition, replacement, and 
governance. When asked what the primary intent of the charter legislation was, Mark 
Roosevelt, a Massachusetts senator and the principle architect of MERA, stated that, 

I supported all [those ideas]; I would have supported vouchers in a millisecond if 
I thought it would help. What we had wasn't working; charters were a push in 
the right direction. . .(Herdman, 2000, p. 39). 

In short, Roosevelt concluded, "There is no clear intent [in the charter law] because there 
were many intents" (Herdman, 2000, p. 39). These multiple intents made the passage of 
the law possible, but muddied the water for authorizers attempting to implement the 
law. 

The different political contexts in the two states colored how the laws were constructed. 
Massachusetts had a stronger labor lobby, and the charter concept was not a prominent 
part of MERA. In contrast, Texas, as a right-to-work state, had a weaker organized labor 
lobby and a mandate for a "strong" charter school law. This difference in political 
contexts became evident early in the growth of these two initiatives. In the first years of 
their respective charter initiatives, both governors asked for a complete lifting of the 
caps on charter schools in their state-of-the-state addresses. Governor Weld was forced 
to settle for incremental growth in the Massachusetts initiative, while Governor Bush 
very nearly got his wish. 

Both states began their initiatives with roughly the same number of schools -- 
Massachusetts had 15 in 1995 and Texas had 19 in 1996, but the number of charter 
schools in Texas quickly surpassed the number in Massachusetts (Figure 6A, below). In 
March 2001, Texas had more than three times the number of operational charter schools 
as Massachusetts. Texas had 167 open-enrollment charter schools and 25 campus 
charter schools, 22 of which were in Houston (Stevens, 1999).^ About half (83) of the 
open-enrollment charter schools were for at-risk students. In the 2001-2002 academic 
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year, Massachusetts had 36 Commonwealth and six Horace Mann charter schools in 
operation, for a total of 42 (MDE, 2001) ^ 

Figure 6A: Charter Schools in Massachusetts and Texas. This figure 
summarizes the growth in the number of charter schools in Massachusetts and 
Texas from 1995 to 2000. 
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Sources: Massachusetts Department of Education, Texas Education Agency, and the 
Center for Education Reform, www.edreform.com. 



SETTING THE TERMS: Getting It Right from the Outset 



Authorizers faced a steep learning curve in developing their charter school applications. 
With its tight limits or "caps" on the number of its available charters, the process in 
Massachusetts was fairly stringent; just one in four applicants was accepted over the 
first six years charters were granted. However, even with a fairly high bar. 



er|c best copy available 
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Massachusetts' authorizers quickly acknowledged that there were charters granted in 
1996 that would not have passed muster in 2001. 

Just as in Massachusetts, the Texas application review process became more 
sophisticated over time. John Stevens (1999), the co-director of the Texas Business and 
Education Coalition (TBEC), chronicled the application process and wrote that in 1996 
"the first 20 charters had been issued to applicants that m.et minimum selection criteria 
on a first-come, first-served basis" (p. 6). Stevens stated that the TEA had received just 
23 applications when the first 20 charters were granted. 

In 1997, the process was improved. Texas' Charter School Division created a budget 
template, a scoring rubric for the ratings process, and a process for training and 
recruiting external reviewers (Stevens, 1999). In March 1998, the State Board of 
Education applied this revamped process to the 86 applications it received and awarded 
charters to the top 41 applicants. 

However, later in 1998, political pressure to expedite the review process caused the 
Charter School Division to change its review process. Stevens (1999) writes that there 
was "considerable pressure on the Board to fill all 120 open-enrollment charter slots and 
approve at least some at-risk charters before the Legislature reconvened in 1999" (p. 8). 
As a result, the State Board of Education (SBOE) moved up the next round of charter 
applications from the spring of 1999 to the fall of 1998. At that September 1998 meeting, 
the SBOE granted 84 of the 109 charters it received (58 open-enrollment and 26 at-risk 
charters).^ Before the legislature convened in 1999, another 25 at-risk charters were 
awarded, for a total of 109 charters. The impact of this rapid growth was that the Texas 
Charter School division had to compromise its review process in order to put forward a 
high volume of charters. Specifically, the Charter School Division suspended the 
interview process. 

After a significant jump in the number of charter schools in the 1998-99 academic year, 
the pace slowed and the authorizers returned to the interview process they established 
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in 1997. The SBOE realized that the rapid pace was outstripping the Charter School 
Division's available resources. At their November 1998 meeting, the SBOE actually 
voted "to ask the legislature to limit the Board's authority to grant more charter requests 
until evaluations could be made of the charter schools already approved and operating" 
(Stevens, 1999, p. 8; also Hood, 1998). However, the legislature did not grant that 
request and the SBOE continued to grant more charters after 1998, albeit at a more 
moderate pace. In 2000, 40 more charters were granted (14 open-enrollment and 26 at- 
risk charters). 

The Importance of Faculty Buy-in 

Once schools are awarded a charter they are given an opportunity to revamp their goals 
based on the population of students they enroll. This can be a formal amendment to a 
school's charter and in Massachusetts this is called the "accountability plan." 

In theory, policymakers might reasonably assume that by providing schools with more 
local control they would use this freedom to set goals that are mission-driven and 
arrived at collaboratively. My data suggested that five of the six schools in my sample 
did have clear missions (Chapter 4). However, my data questions whether the goal- 
setting process in these schools was a) collaborative and b) informing the everyday 
operation of schools. 

Massachusetts charter schools are required to complete an accountability plan outlining 
their organizational goals at the close of their first year of operation. Rather than a 
committee of parents, teachers and administrators poring over their school's goals, a 
board member of MA-A suggested that one or two people generally set the goals and 
the rest of the school often has little input. She described the process thus: 

We got sat down and told that we had to have an accountability plan and we 
were told that we had to have school performance objectives, and student 
performance objectives... there had to be two of that, and three of that.^ They 
couldn't be things that were maybe at the heart of what the school was [or 
things] we were already well on our way to achieving. They had to be things we 
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had not yet accomplished. So already this is a little bit of a weird document.... Is 
this a format that made sense to anybody in this school? No. [But] this was the 
state requirement. Now is it something that people really care about and take 
seriously and really think they are measuring themselves against? I'm not sure if 
anybody looks at this but me and [other board member] when we write the 
annual reports. 

This finding was supported in my visits to the other Massachusetts and Texas charter 
schools. That is, when teachers or parents were shown the site-visit reports or 
accountability plan of their school, the vast majority had not seen the document before. 

In sum, while the process of goal setting in itself may have been a helpful process at the 
outset, it was not clear that the accountability plans were living documents that 
continued to bring these schools together around a common set of expectations. 
However, as will be described below, while these goals appear to have little value in 
isolation, they do appear to have value when coupled with an on-site visit designed to 
evaluate the school against its goals. 



GATHERING THE DATA: The Power of Combining Tests and On-Site Reviews 

If government priorities can be gauged by what is collected; compliance data is 
important. Compliance is and should be a primary function of an authorizer. This 
capacity and need for compliance monitoring is reflected in the data. The USDOE (2000) 
reports that, in 1998-99, charter schools in a selected sample most frequently reported 
monitoring in the areas of school finances (94%), compliance with state or federal 
regulations (88%), student achievement (87%), student attendance (81%), and 
instructional practices (63%) (see Figure 6B, below). 
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Figure 6B. Charter School Monitoring. This figure summarizes the types of 
monitoring conducted by a select group of states across the United States. 
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Source: USDOE 2000 



Fiscal and civil rights monitoring are crucial responsibilities of authorizers and the most 
frequent forms of government monitoring (Figure 6B). Because state education agencies 
(SEAs) are responsible for overseeing federal funds under civil rights law/ the state is 
ultimately liable for the misuse of those funds.^ Further, because 79 of 86 (or 92%) of 
the closures that have taken place as of December 2000 are due to compliance or fiscal 
issues (Center for Education Reform, 2000), such monitoring is needed and warranted.® 



Measuring School Performance: Tests Versus Site Visits 

Measuring performance is becoming increasingly popular among policymakers, but the 
science of measuring school performance is less developed and far more subjective than 
auditing finances or verifying whether policies are being followed. 

While there are systems in place to monitor fiscal and regulatory compliance, measuring 
student and school performance is still very much a nascent science. Across the country, 
49 states, with the exception of Iowa, now have academic content standards (more than 
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three-fold increase from the 14 states that had such standards in 1996). There is a 
continuum across those states as to how much state agencies rely on test scores versus 
other measures, such as site visits, when evaluating school performance. Texas is closer 
to the test-reliance end of the spectrum and Massachusetts (as it relates to charter 
schools) sits closer to the site-visit end of the spectrum. Both approaches have their 
strengths and weaknesses. 

In examining the responses of the six schools in my sample, three findings emerged: 

• Tests had a positive effect on low performers and little effect on medium and high 
performers; 

• Site visits had a positive effect on schools of all performance levels 2 / they were 
customized to the unique expectations of each school. 

• Tests and site visits provide important and complimentary information. 

Tests Have Greatest Effect on Low-Performing Schools 

While the MCAS is generally seen as a more challenging test than the TAAS, the 
responses across the six schools in my sample were relatively consistent by performance 
level.^ In general, the two lowest performing schools (TX-C and MA-C) appeared to be 
more responsive to their state tests. 

Teachers at TX-C claimed they did not "teach to the test," however, the principal of the 
upper school did pay close attention to how well students performed as individuals and 
within grade levels. The principal had a chart on each individual student and had 
highlighted which items that student missed. In response to a question, "How do TAAS 
scores effect what happens in your classrooms?" he said: 

We dissect them. I did that last summer. I look at each individual and I can 
show you what happened. So, a teacher can come in here and look at all the 
students and make a chart of the objectives on the TAAS that were not met by a 
majority of the students. These are the objectives on the TAAS for the 9^^-grade 
[presents a chart of skills for each student with deficiencies highlighted in 
yellow]. 
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The reaction to MCAS scores was more significant in MA-C the lowest-performing 
Massachusetts charter school in my sample. In 1999, none of their MA-C fourth grade 
students passed the English Language Arts portion of the test and the school as a whole 
performed worse than any school in the surrounding district. In response to the same 
question asked above - "How do MCAS scores affect what happens in your 
classrooms?" - the principal responded: 

A lot. We got killed this year; part of what I found out is that we weren't doing a 
good job preparing our students for the MCAS last year. The good thing about 
[MA-C] is that we don't try and sugar coat the results to parents. Parents, 
particularly Black parents, know the research out there but we also don't try to 
hide behind the research. We don't try to say, "Well you know we're only a 
Black school, how do you expect us to do well in MCAS or anything?" We don't 
take that position at all because I don't really believe that. 

In 2000, MA-C completely revamped its curriculum, hired a curriculum director, and 
sought out technical assistance to improve its instructional focus. 



State Tests: A Default or Distraction for Medium- and High-Performing Schools? 

Generally, charter school leaders and teachers in medium- and high-performing charter 
schools tended to view state tests as everything from a benign frame of reference to a 
distraction from meaningful learning. No school is monolithic in its views; however, 
those interviewed in MA-A, MA-B, TX-A, and TX-B did not appear to be "teaching to 
the test" as some have feared (Orfield & Wald, 2000). 

There were, however, state level differences in response to state tests. Most Texas 
educators saw the Texas state standards (Texas Educational Essential Knowledge and 
Skills, TEEKS) and state tests (Texas Assessment of Academic Success, TAAS) as a 
minimum hurdle. For example, TX-B, the middle-performing charter school in Texas, 
used the TAAS as a frame of reference but did not focus on it in class. A TX-B teacher 
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said, "We don't drill and kill, but we do look at the TEEKS to make sure we are on task." 
Similarly, one TX-A teacher said he found the TEEKS helpful guidelines, especially since 
he was from out of state. However, he was clear in pointing out that the Texas 
standards were the "minimum expectation" and that the school covered them by 
[default]. He said: 

We look at the state minimum first and then we see how we can incorporate the 
I[nternational] B[accalaurate] objectives.... Any school would probably not teach 
to the state minimum expectation, which is essentially what they are. We end up 
feeling that if we teach our IB objectives, our text will get covered and TAAS is 
practically a default. 

The Massachusetts charter school faculty tended to be more frustrated with the 
Massachusetts Comprehensive Assessment System (MCAS) than Texas charter school 
faculty members were with the TAAS. The views of educators in MA-B and MA-A on 
MCAS ranged from mild annoyance to vehement objection. For example, a curriculum 
director in MA-B found that MCAS did not align with the educational management 
organization's curriculum. She had more faith in the EMO's math sequencing than the 
state's. Therefore, she felt a tension between a) reorganizing the curriculum to match 
the state's expectations, which might have a deleterious impact on students, or b) 
maintaining their current sequencing and risk performing poorly on the state tests. She 
described her situation this way: 

The MCAS is more of a hindrance than a help to us. We have a curriculum that 
we think works and the order in which the Massachusetts math standards are 
presented does not align with our curriculum. As a result, in order to make sure 
our eighth graders do well on MCAS, we may be sacrificing [our students'] 
ability to be competent in Algebra by the ninth grade, a foundation we feel is 
necessary for the upper level math courses they will need for college. 

MA-B ultimately changed the order in which they presented algebra classes to align 
with the state standards. Thus, in this case, one could argue that MCAS may have had a 
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deleterious effect on the school's internal accountability in that it compromised the 
school's expectations in regard to good teaching and learning. 

A social studies teacher in MA-A was probably the most adamant about his resistance to 
the MCAS. He entered the interview with an anti-MCAS button on his lapel and when 
asked about the button he said: 

The Massachusetts Social Studies standards miss the entirety of those 
recommendations and international courts we used to talk about. They are 
almost directly counter to our ability to teach thinking skills. If you set out to 
cover that much material, you can only do it superficially, through certain kinds 
of pedagogy. I know they suggest to us, well, we're creative teachers we can find 
ways to do it, but on the face of it, it's pretty counterintuitive. 

Even though MA-A students performed better than almost every other school in the 
state, they organized rallies at the Massachusetts statehouse against the use of MCAS as 
a graduation requirement. 

The Potential of Value-added Measures 

A final point regarding test scores was that the lack of emphasis placed on "value- 
added" test scores was a common concern among charter school educators in both 
states. That is, the school personnel believed that schools should be evaluated based on 
how their students progressed relative to their baseline or first-year scores. Both states 
have value-added measures, but in both states a single score is most commonly 
reported, i.e., percent passing in Texas and a scaled score in Massachusetts. When asked 
how his definition of success agreed or disagreed with the state's definition, a TX-C 
principal responded this way: 

The state defines [success] as TAAS scores. I personally would like to see 
measurement as improvement, and not necessarily on TAAS, just on 
improvement. They've got the TLI, Texas Learning Index, which tracks student 
performance [from grade to grade], which I think is good. But to me that's 
overshadowed by the group TAAS scores, where you only had 87% passing the 
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math... so you're an "acceptable" school or you're this. I have students coming 
here in the 9^*^ or 10^*^ grade reading at the 4^*^- or 3 ^^- grade reading level.... [If] you 
want me to have them pass the 10^*^-grade TAAS because they are in the 10*- 
grade, but they're reading at the 4*-grade level - you want me to work a miracle. 

If yearly testing becomes the norm, as mandated in No Child Left Behind Act of 2002, 
the inipor lance of value-added assessments m.ay play a more prominent role in 
measuring student performance. 

In sum, my findings suggest that schools identified by the state as low performing were 
the most responsive to the respective state tests (MCAS and TAAS). Based on a sample 
of six schools, it is impossible to generalize and assert that all charter schools in 
Massachusetts and Texas will act in a similar manner, but it stands to reason that they 
might. In theory, the greatest consequence the state can mete out — charter revocation — 
will befall the lowest performers, it is rational, therefore, that these schools would be the 
most responsive. This finding supports the work of Hoffman et al. (in press), who found 
that teachers in low-performing Texas schools reported greater frequency of test 
preparation than did teachers in high-performing schools. Using this same logic, it is 
understandable why medium- and high-performing schools are the least concerned with 
these tests. Without the threat of sanctions, these schools have the option of largely 
ignoring the state tests. 

An explanation for the more agitated response from Massachusetts educators might be 
due in part to the construction of the MCAS. MCAS covers more subject areas than 
TAAS (i.e., social sciences, and science are tested in Massachusetts and are not yet tested 
in Texas), and MCAS is generally perceived to be a more difficult test than TAAS. It 
stands to reason that there is likely an inverse relationship between the level of freedom 
a school has in developing its curriculum and the rigor and breadth of a state 
assessment. 
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Did test scores affect the organizational capacity of these six schools? The low- 
performing schools in my sample were the most responsive. In particular, MCAS scores 
inspired MA-C to reexamine its curriculum, which in turn benefited the professional 
knowledge and skills of the teaching staff. However, beyond the effects in MA-C, there 
was a negligible positive impact (and in the case, of MA-B, possible negative impact) on 
organizational capacity across the other five schools. Overall, the test scores among the 
schools in my sample most likely had more external value as an indicator of 
performance for parents and lawmakers than internal value in terms of positive 
influence on classroom practice. 

Site Visits Are Influential Across Performance Levels 

A second tool used by Massachusetts and Texas authorizers to monitor school 
performance was site visits. While both state agencies had officials visiting the schools, 
the focus of the visits differed by state. The site visits in Massachusetts were customized 
to each charter school. The intent of these visits was to "validate and augment" the 
annual reports of each school. In contrast, the visits in Texas were primarily compliance 
oriented, e.g., to ensure that special education dollars were being spent in accord with 
federal civil rights regulations. The stakes were also different in the two states. Without 
an established state testing system, the first generation schools in Massachusetts knew 
that these site visits would carry considerable weight. In contrast, the Texas schools in 
my sample were likely more focused on the TAAS. 

Massachusetts' effort to align the state's expectations with the school's expectations led 
the schools in my Massachusetts sample to use these site visits to improve their 
organizational capacity In contrast, the Texas site visits did not appear to influence 
the organizational capacity of the charter schools in my Texas sample. 

Texas Visits More Compliance Focused 

An excerpt from Texas Education Agency (TEA) correspondence sent to TX-B following 
a site visit highlights the compliance orientation of the visit. It states: 

[T]he visit consisted of a formative evaluation. [I]ts purpose was to give the 
charter school a course of action and direction for the future so that the student 
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population served can be provided maximal services within the boundaries of 
state and federal laws. When the charter school becomes a part of the 
monitoring cycle for District Effectiveness and Compliance (DEC) visits, it will be 
expected to comply with rules and regulations specific to special education and 
to bilingual education and English as a Second Language (ESL) programs. 

While the site visits are conducted by TEA staff in Texas as comnared to a team of 
private citizens led by a MDE staff member in Massachusetts, the site visit protocols are 
similar. Much like the Massachusetts site visits, Texas site visits involved focus groups 
with parents and teachers. Reports documenting the visits note whether the school was 
doing what it had promised in its charter. For example, the TX-B report noted whether 
the school was using a national reading program as agreed to in its charter. TX-A 
principal said she appreciated the chance to reevaluate where they were. She said: 

One thing we've been encouraged to do by the state, which I think is really 
important, is to "walk the talk." [They said] to look at your charter really 
carefully, if there is something you really don't know - or a lie got in there, or 
you don't believe in it anymore, or if you tried it and it just didn't work, this is 
the time to really clean it up and re-draft it. 

However, beyond perhaps improving their record-keeping systems, these site visits did 
not appear to impact classroom instruction or school governance. When asked how the 
TEA site visit has influenced what happens in TX-C classrooms, the upper school 
principal responded: 

They look at our records, special ed., how do we manage special ed. We're still 
under federal guidelines.. .So they're just looking to make sure those things are 
complied with, which is good. 

Despite several direct questions about classroom or school impact, none of the educators 
or principals at the three Texas charter schools in my sample provided an example of 
how the state's site visits informed their practice (beyond improving their compliance 
systems). 
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Massachusetts Visits More Focused on Performance 

Massachusetts' site-visit reports are far more subjective than the Texas site-visit reports. 
In Texas, the visitors reported on whether the school was offering the services it said it 
would. In Massachusetts, the site visitors commented on the quality of the services 
provided based on the schools goals and a state developed rubric. For example, the 
following is the conclusion of the MA-B site visit report in 1997: 

The [MA-B] charter school has developed what appears to be an excellent 
education program that is based on high standards, is rigorous and challenging 
for students, and is dedicated to the academic progress and success of each 
student. The Board, staff, students and parents all appear to be highly 
committed to the academic mission of this program. The site visit team was very 
impressed by all that the school has accomplished in such a short period of time. 

The stability of this strong education program, however, is threatened by 
the apparently severe breakdown of communication between the school's Board 
and administrative team. Moreover, both the Board of Trustees and the 
administrative team (and by extension, the school) suffer from a lack of clarity 
and agreement regarding their respective roles in and responsibilities for the 
governance and operation of the school. 

In Massachusetts, schools at all three performance levels could readily cite examples of 
how the site visits (either the single-day annual visits or the three- to five-day renewal 
inspections) helped improve their organizational capacity. A board member (and 
former principal) from MA-A was impressed with the site visitors' ability to review the 
school on its own terms. He said: 

I would call it genuinely helpful. They didn't understand some things,^^ but they 
understood quite a lot. And they pursued things to depth, as far as I could tell 
they really did their homework.... They did what they said they were going to 
do, which was take us at our word. 

When pressed for specific examples of how the site visit may have informed classroom 
practice, the board members cited several examples. They suggested that the site 
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visitors helped the staff rethink how it plans curriculum and that several teachers saw 
the visit as a valuable professional experience.12 For example, one board member 
commented, "Our science teacher said she had the most valuable professional 
conversation she ever had in her career." 

However, the main benefit may have come before the site visitors ever entered the 
school. A board member said, "I think getting ready for the inspection itself was 
perhaps 70% of the benefit. [We'd say,] 'Oh my god, we're really not clear on that are 
we?' Then the interchange would begin." Echoing this sentiment, a MA-A board 
member, who was a self-proclaimed "cynic," argued that the validation that comes with 
the site-visit process is valuable in and of itself. She said: 

I can't honestly say that we've ever learned anything from a site-visit report that 
we didn't already know. This is a community of people who are "reflecting" all 
of the time.... [However,] I think it is important to have these site visits, so don't 
get me wrong on that — but not because you learn anything from them. It forces 
you to stand back, think about where you are, make presentations, and if 
everything is going well in your school you probably won't learn 
any thing... because it will support what you've already discovered about 
yourself.... [But it] begins a conversation that typically validates that, "yeah, 
we're not crazy." 

At MA-B, there were two significant changes as a result of site visits. Comments in the 
1997 site-visit report (cited above) called on the board to clarify its roles and 
responsibilities vis-a-vis the EMO. In essence, the report stated that the governing board 
was micro-managing the day-to-day operations of the school to the detriment of the 
students. Board members said they "backed off" as a result of that report. In fact, the 
contract between the board and the EMO now stipulates that all employees of the school 
are employees of the EMO, so if the Board is dissatisfied with an employee's 
performance, then they can go to the EMO. If, over time, the Board is dissatisfied with 
the EMO, the Board can fire it. However, the Board is not involved in the daily 
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decisions of the school. This has dramatically simplified the roles and responsibilities 
among the EMO, the principal, and the Board. 

Second, the principal of MA-B said that they changed how they were supporting special 
needs students as a result of a site visit. She explained that the school has short-term 
"intensives," or remedial classes, for students who are falling behind in their math or 
English. However, a review of student records revealed that special needs students 
were languishing in these classes. She explained that this was "totally wrong" and so 
the intensive teacher, the special education intensive teachers, the special education 
coordinator, and the principal are realigning these programs for 2001. 

At MA-C, the school completely reworked its curriculum at least in part because of the 
site-visit report in 1997. (Recall that this site visit took place before MCAS was 
administered.) The MA-C site-visit report concluded: 

The school still has considerable weaknesses to address. The education program 
is not clear. The school does not have explicit, school-wide goals for its students 
in the curricular areas specified in its charter and accountability plan. Moreover, 
the school has presented no evidence of the academic progress for its students 
during its first year, nor has it developed a clear plan for measuring student 
progress beyond the administration of standardized tests. Without developing 
clear academic goals and tools to credibly measure student progress relative to 
such goals, the Board of Education will have no record of performance on which 
to base such a decision to renew the school's charter. 

A MA-C teacher acknowledged that the site-visit report was a catalyst for the school's 
effort to rethink its entire curriculum: 

When I first got to the school and [the principal] hired me, he told me about the 
site-visit [report]. I remember him reading it to me. He actually asked me a 
question in the interview about it. [My response was], "It seems to me that you 
just need some more internal means of assessment and it has to be consistent 



94 

Herdman 



101 



between now and the next time they come back." He wanted to implement a 
curriculum that was aligned and clear - which is ultimately what we did. 



In sum, how do site visits impact organizational capacity? In Texas, the site visits 
appear to have helped the three schools in my sample reassess how well they were 
complying with civil rights law and gave the schools an opportunity to refocus their 
charter. However, the visits did not appear to challenge the schools to reevaluate their 
performance .^3 

In contrast, the Massachusetts visits resulted in positive changes in three of the four 
components of organizational capacity. MA-A provides evidence of improvements in 
teacher professional knowledge and skills. MA-B offers an example of how the site 
visits may have improved the school's leadership structure. MA-C provides an example 
of how the site-visit process lead to a school revisiting its mission and reevaluating its 
instructional approach. 

These findings lend support to the work of Abelmann and Elmore (1998) that external 
accountability will have a greater effect if it is aligned with the internal expectations of 
the school. As a component in these incentive systems, site visits send a signal that the 
authorizers are concerned about the organization as well as it delivery of services. 

USING THE DATA: The Challenge of High Stakes Decisions 

Once the data are gathered, the next question is what to do with it. An authorizer can 
make one of three decisions: 1) reward a school for strong performance; 2) provide 
assistance to a school that is struggling; or 3) sanction a school by putting it on probation 
or closing it. These decisions can be made at various points during the five-year term of 
a school; however, the charter renewal decision usually only occurs at the end of that 
five-year term.^^ 
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The most severe sanction in either state system is revocation of a school's charter. In 
Texas, TX-A and TX-B had their charter renewed in the winter of 2000-01, and TX-C, the 
lowest performing school in my sample, imploded on its own. It was managed poorly, 
it was $1.5M in debt, and closed its own doors in the fall of 2000. The state did not have 
to make a judgement about whether its charter should be renewed. 

As discussed in chapter 5, all of the Massachusetts schools went through their renewal 
visits in 2000 and all 14 of the first-generation charters were renewed.^^ 'Yhe decisions 
were made based on a mix of financial, compliance, test score, and site visit data. As has 
been discussed above, MCAS did not come on line until 1998, three years after the first 
schools became operational. Therefore, the test data on many of the first- or second- 
generation charter schools is a mix of some state level data and school-selected, norm- 
referenced tests. Not only is the data set incomplete, but most of the data is not 
disaggregated by race or income. As a result, the most consistent data is likely the site- 
visit reports. 

Despite their value as a formative evaluation tool, site visits present at least three 
problems as a summative evaluation. First, their reliability is questionable in that 
different visitors participate in each site visit so the expectations from school to school 
are not consistent. Second, while a one-day or a multi-day visit provides a much richer 
understanding of a school than test scores might, it is unclear whether these snapshots — 
in isolation — are valid assessment of a school's performance. Third, the Massachusetts 
site-visit process had problems with transparency. "Academic success" was never 
defined, so the questions of, "how good is good enough?" and "how bad is bad enough" 
were never made clear. A charter school leader said of his school's site-visit report. 

It provided us with a tremendous amount of great information, but I wasn't sure 
where we stood. I mean, it said we did well on some things and poorly on 
others, but what did that mean? Were we doing great compared to the other 
charters or were we on the brink of getting shut down? 
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In short, the process was not transparent in that the expectations were not clear at the 
outset, and there was not adequate notice built into the system so that schools could take 
corrective action. The ambiguities in the site-visit data make for a shaky foundation on 
which to build a high-stakes accountability system. 

Despite these challenges, Massachusetts" authorizers had to rely on this measure to 
make such a decision under extreme time pressure. MA-C exemplifies the challenges in 
this oversight system. 

MA-C: A Charter Renewal Case Study 

In March of 2001, MA-C went in front of the Massachusetts State Board of Education. 
MA-C had exceptionally low test scores and a seemingly dysfunctional board. In fact, 
when reflecting on the school's performance, a former academic advisor to the school 
said, "By all rights that school should have been closed." So why did the Massachusetts 
SBOE, the "Harvard of charter school accountability" (Finn et al., 2000), vote to renew 
this charter and what are the implications of that decision? 

Recall that the Massachusetts charter school accountability plan is driven by three 
questions: 

• Is the academic program a success? 

• Is the organization viable? 

• Is the school faithful to the terms of its charter? 

Is the academic program a success? MA-C students performed well below state averages 
on the fourth-grade MCAS in 1997 and 1998. In 1999, none of the fourth-grade students 
were proficient in English Language Arts; 6% were proficient in math; and 3% were 
proficient in science and social studies. MA-C also performed well below the district 
averages (the lowest in the district). In addition, site visitors reporting on classroom 
observations noted that in "some cases, students were engaged in low-level, rather than 
challenging, academic activities" (MDE Renewal Findings). 
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Is the organization viable? The board has had high turnover and its roles and 
responsibilities vis-^-vis the school's leader are unclear. The renewal findings state. 



Since the school's founding in 1996, there has been substantial turnover in the 
Board of Trustees membership. Only one of the founding members remains on 



the board today, and the board membership is significantly below the number 
prescribed in the by-laws.... While individual members articulate a commitment 
to continued improvement of the school and have ideas about how to proceed, 
these ideas are not organized in a collective vision or plan. Staff members 
remain unclear as to the role of the Board in the leadership of the school. 

Is the school faithful to the terms of its charter? The focus has changed from an Afro-centric 
curriculum to a more standards-based curriculum, but still concentrate on working with 
minority students in the areas of math and science. The school has not been cited for 
any fiscal or procedural malfeasance. So, in essence, it has remained faithful to the terms 
of its charter. 

In short, if the rhetoric of performance-based accountability were strictly adhered to in 
this case, the school might well have been closed. The mission of the school has 
endured, but the school's student performance and organization's viability are highly 
questionable. 

So why was the school's charter renewed? Was this a political decision— based on a fear 
of closing down a predominantly African-American school that had very supportive 
parents and the backing of prominent Democratic legislators, or was there evidence that 
the school had more public benefit in remaining open than closing? One of the issues 
that my review of the data above misses is the trajectory of the organization. For 
example, while MA-C performed poorly on MCAS, an examination of Stanford-9 scores 
in grades 1 through 8 in reading, language, and math reveal improvement trends. 
Further, and perhaps more importantly, cifter a year of having no curricular focus, a new 
principal was hired who renewed the school's focus on academics and rewrote the 
school's entire curriculum. The renewal inspection team noted: 
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The school's executive director has provided skillful leadership in his four-year 
tenure. Early on he identified three overarching barriers to the school's 
achievement of its mission: a) lack of a sound academic program. . .b) significant 
student behavior problems; and c) lack of effective leadership. Since that time, 
he has enabled the school to make significant progress in each of these areas. 

In short, the site visit report indicates that there were significant changes in the 
organizational capacity of the school that may have warranted the school's charter 
renewal. 

However, as is usually the case, the data is not clear-cut. This was a difficult decision, 
and perhaps it was the correct one, but it raises the specter of how good is good enough 
for charter renewal. Further, short of malfeasance, what is low enough to warrant 
charter revocation? And while the trajectory of the school may be positive, this director 
has been in place for four years; how long should policymakers wait to see improved 
outcomes? For example, despite the generally positive tone of the renewal inspection 
report, it notes that the "shift to a standards-driven learning environment is currently 
stronger in school-wide planning at [MA-C] than in daily classroom planning." 

A member of the Massachusetts charter school office called for more specificity in the 
state's process and wanted to see a clear recommendation in the state's renewal 
inspection reports. He said: 

MA-C was on the line, their MCAS scores were low, but they were making 
improvements. The commissioner asked what we should recommend and I said 
we should renew the charter with conditions.^6 in the absence of clearer 
standards, we ran the risk of undermining the Board [of Education's] credibility. 
If we recommended that they revoke the charter, there is a good chance that we 
would have lost in arbitration. The renewal inspection report in an effort to be 
balanced does not make a clear recommendation. 

This decision also highlighted how the lack of capacity in the authorizing office adds to 
the challenge of holding schools accountable for performance. The authorizer explained 
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that SchoolWorks, the company contracted to the renewal inspections, did not want to 
make a summative evaluation of the school based on one three-day visit. In fact, the 
regulations (CMR.603) state that the renewal inspection report should be considered as 
one piece of evidence in conjunction with the school's annual reports, audits, test scores, 
and other site visits. However, it is understandable why the authorizer wanted more 
help when he explained his timeline leading up to the MA-C renewal decision. 

Mapping backwards, he explained the limited time he had to dedicate to this process. 

He said: 

... I had to get a draft to the legal staff and the other members of the charter 
office in early January. The renewal visit took place in December, so I had a 
couple of weeks to review all of that data and write a report. And at the same 
time, we had to review new charter applications, four other charters were up for 
renewal, and North Star charter school in Springfield was in the process of losing 
its charter, so I had to spend an inordinate amount of time with them. 

No state oversight system will (nor should) be completely devoid of subjectivity. 
However, a lack of clarity from the outset and an ambiguous feedback process make the 
threat of charter revocation remote. The issue then becomes, if renewal becomes rote 
and the threshold for charter revocation becomes so low that only the most criminally 
poor performing schools are shut down, what bar does that set for the charter school 
movement? What impact will that perception have on all of the incremental steps 
leading up to that juncture? Without the threat of consequences, will charter school 
leaders invest energy and time into accountability plans and site visits? 

A DISCUSSION OF CUSTOMIZED VS. COMPARABLE DATA 
What are the lessons learned in the charter renewal decision of MA-C? As Table 6A 
summarizes, there are strengths and weaknesses associated with over-reliance on either 
state test scores or site visits. Test scores are a more valid and reliable external measure 
and they are less costly and time-consuming than site visits. However, an over reliance 
on test scores will likely work to erode a charter school's efforts to be more creative, i.e.. 
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common tests could push charters back to the mean. This tension between the need for 
comparable data and limiting teacher freedom is likely the most apparent in social 
studies exams because, unlike language arts and mathematics which tend to be more 
skill oriented, this domain is far more subjective. How history is defined in a state's 
standards can have a significant influence in limiting the historical interpretations of 
classroom teachers. This all suggests that state tests are necessary measures, but high 
stakes consequences should be tied to a narrow group of subject areas, e.g., numeracy 
and literacy. 



Table 6A. Summary of Site Visits versus State Tests. This table summarizes the 
strengths and weaknesses of site visits and state test scores as measures of school 
performance. 



INCENTIVES 

Comparable 

Tests 


Customized 
Site visits 


External: Reliability and validity as a 


strong 


limited 


summative tool 






Internal: Value as a diagnostic and 


limited 


strong 


formative tool 






School buy-in 


limited 


stronger 


Potential for improving organizational 


limited 


stronger 


capacity 






Time 


minimal 


high 


Cost 


lower 


higher 



Site visits provide more internal value as a formative assessment than state tests do. 
Further, because they are designed around a school's own goals, they have the potential 
to generate more school buy-in and as a result, have a greater impact on a school's 
organizational capacity. The Massachusetts and Texas oversight systems exemplify the 
complimentary strengths and weaknesses of state tests and site visits. This suggests that 
if policymakers want to have a system that is valid and transparent as well as one that 
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encourages all schools to continuously improve, then a combination of comparable and 
customized should be considered. 

While these authorizers have made strides in gathering helpful data, enforcing 
consequences based on these data has been challenging. The ability of an authorizer to 
close an under performing school is arguably the lynchpin of an accountability system. 
While some charter schools have been closed in both states, none have been shut down 
for poor performance alone. Thus, the bar has been set at just above the minimum. The 
question then is how will this precedent influence the accountability process as a whole 
and how might these systems be strengthened in the future? Certainly setting clearer 
terms of agreement at the outset is an important first step. 

The next, and last, chapter will summarize my findings and discuss the implications of 
this work. 



^ Herdman (2001), Unraveling the Basic Bargain: A Study of Charter School Accountability in 
Massachusetts and Texas provides in-depth case studies of how the charter school laws and 
policies were constructed. 

2 The other three campus charters are in Dallas, Nacogdoches, and Spring Valley (Stevens, 1999) 

3 As of June 15, 2001, Massachusetts has granted 48 charters, seven in the spring of 2001, and 40 
charter schools are operational. 

^ There is a discrepancy in the numbers. Stevens (1999) reports that 59 open-enrollment charters 
were awarded at the September 1998 SBOE meeting. TEA data received in April 2001 indicates 
that 58 open enrollment charters were awarded that year. 

5 MDE hired consultants to work with each school individually to develop their accountability 
plans. 

^ Civil rights laws pertain to federal laws such as the Individual with Disabilities Education Act 
(IDEA), Title II of the Americans with Disabilities Act, and Section 504 of the Rehabilitation Act 
of 1973. 

^ Personal communication with Tom Hehir, former director of Special Education Services, 



® Center for Education Reform divides these 79 closures across three groups: finances, 
mismanagement, and facilities. However, all three groups are related to the availability and 
management of finances. The remaining 8% of closures are for reasons defined as "other" (1%) or 
"academic" (7%). 
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9 Within schools different teachers had different perceptions of the state tests depending on the 
subject area they were teaching. For example, the math teachers were less concerned by the math 
standards because there is a fairly common understanding of the expectations in this subject. 
Conversely, social studies teachers were more resistant to state standards in Massachusetts 
because there is a greater variation in the thinking on how this subject should be taught. 

See reference to Abelmann and Elmore (1997) in Chapter 2 for a more detailed discussion of 
alignment. 

A Social studies teacher at MA-A complained that the lead inspector, who was a historian, and 
contributor to the Massachusetts frameworks, was trying to force him to present history in a 
chronological format that did not match the school's pedagogy. 

After observing some of their curriculum planning sessions, a site visitor noted that it was 
some of the most thoughtful discussions he'd heard in a teacher planning session. However, he 
questioned whether the teachers could improve the quality of the instruction if they planned the 
curriculum more proactively, e.g., over the summer, rather than week-by-week. [This was 
relayed by a board member of MA-A.] 

None of the schools I visited had a "risk analysis" visit from the TEA. The schools in my 
sample all described their DEC visit. 

Massachusetts charter schools can opt to be reviewed earlier than five years in the event they 
want to have a longer charter in the second cycle. 

YouthBuild Boston Charter School was one of the first 15 charter schools and it returned its 
charter in 1998 due to a range of fiscal and academic deficiencies. 

MA-C did need to agree to four conditions relating to improving the training of their teachers, 
increasing MCAS scores, developing internal and external assessment systems and improving the 
training of their governing board. 
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Chapter 7: Lessons Learned 



This paper views charter schools as a case study in public school governance. There is 
growing agreement that the solution to those problems will come, if only in part, by 
improving the balance between school-level autonomy and central accountability -- 
what I refer to as the ''basic bargain". Charter schools represent an extreme on that 
continuum of local and central control and, as such, provide an important point of 
reference for policymakers involved in the governance of public schools. 

An overarching question in this paper was, Vslhai can charter schools teach us about the 
governance of public education? As my research shows, the charter initiatives in 
Massachusetts and Texas have not "solved" this conundrum of autonomy versus 
accountability. Their policies are evolving and neither states' authorizer has agreed 
upon the best way to oversee this new breed of public schools. However, both the 
challenges and the successes in the Massachusetts and Texas charter school initiatives 
can offer lessons to charter authorizers district superintendents interested in moving 
toward contract or charter-like arrangement with their schools. The following are six 
such lessons. 



LESSONS LEARNED 

Lesson 1: Engage in the Political Process 

The shape of education reform in each city and state across the United States needs to 
run the gauntlet of political battles. Through these battles, alliances form resulting in the 
coupling of seemingly conflicting intents, e.g., the notion that charter schools can be a 
vehicle for both competition and collaboration. The compromises forged in these 
conflicts form the statutory framework and establish the level of discretion available 
within each state's charter school law. Politics shape the parameters of how 
accountability is defined. 
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Both state charter school laws began with a broad array of intents ranging from 
competition to innovation. These multiple intents enabled charter bills in both states to 
garner bipartisan support, but they made interpretation of these laws in the regulatory 
process complex. For example, in the application process, authorizers were forced to 
determine the relative importance of granting a charter to a small school that might offer 
an alternative to the existing options or selecting a larger school managed by an EMO 
that has a track record and could put more pressure on a given district to improve. 

The framework established in the law and the subsequent regulatory process will impact 
the quality of their fellow charter grantees, as well as the level of funding and autonomy 
schools in a given state will have. The key lesson here is that educators should engage 
in the political process. They should look to shape, rather than respond to, the process. 

Lesson 2: With More Autonomy Comes More Responsibility. 

Researchers have hypothesized that if schools were autonomous and mission-driven, it 
would enhance their ability to increase student performance. However, with those 
freedoms came significant challenges. Because these schools were independent from 
their respective districts, they had to shoulder extensive start-up, facilities, and 
administrative costs as well as manage the complex governance issues associated with 
running a public institution of choice (e.g., developing a board, serving all students, and 
meeting the needs of parents). These financial and governance challenges may help 
explain why leader effectiveness and teacher skills were of inconsistent quality in my 
sample. That is, in some schools, the demands that came with being an independent 
public school may have served to make the leadership challenge even more difficult as 
well as served to deprioritize the allocation of resources to the core enterprise of 
schooling -- classroom instruction. 

Despite these challenges, the message to policymakers should not be to reduce school- 
level decision making; this was central to the operations of all of these schools. The 
lesson is that within systems of schools, individual schools should share (among 
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themselves or with an authorizer) the responsibilities of finding and funding a facility as 
well as the onus of providing a full range of special education services. 



Autonomy has its trade-offs. The ability to choose a mission and to teach, hire and fire 
according to that mission, were essential enabling conditions for the schools in my 
sample to succeed, however, autonomy bordered on isolation in some cases. 

Detachment coupled with heavy financial and administrative demands will not lead to 
the creation of high-performing systems of schools. 

Lesson 4: Opportunity Should be Balanced Against Capacity. 

Charter school authorizers face several complex tradeoffs. One of the most critical is 
deciding how much flexibility or opportunity to provide charter schools at the local and 
systems levels. At the school level, the institutions need to have the flexibility to truly 
respond to their communities' needs, but too much local control could serve to outstrip a 
school's capacity to manage itself responsibly. It was clear in my sample that charter 
schools develop at different paces. Thus, oversights systems should be designed to 
respond to variations in school-level capacity. 

At a system level, legislators and authorizers need to decide on the number of charters 
and the means by which they can be granted. In so doing, policymakers face another 
tradeoff. By providing too little opportunity, they may be under-addressing the needs 
of students trapped in low-performing schools, and by providing too much opportunity 
authorizers risk wasting millions of taxpayers dollars on ill-prepared schools. Creating a 
system with too much opportunity may yield a system in which schools are closing 
unexpectedly at high rates and dislocating students. However, clamping down on that 
freedom too hard could mitigate the catalytic effects of market competition and reduce 
the possibility of generating new ideas that could improve the system as a whole. Thus, 
lawmakers should be conscious of building capacity levels within their government 
agencies that match the levels of oversight required. 
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Be aware of capacity at the school and agency levels when phasing in this new system. 

It was clear in my sample that autonomy may have enabled some charter schools to 
develop high internal expectations, but it did not ensure it. Schools developed at 
different rates, and had different levels of capacity to handle the responsibilities that 
came with independence. Similarly, the authorizers needed time to develop appropriate 
oversight systems. In short, systems should change at a pace that is commensurate with 
an oversight body's capacity to manage it effectively. 

Lesson 4: The Culture of Government Oversight Needs to Change 
The Massachusetts and Texas authorizers played critical roles in ensuring equity and 
encouraging excellence in these schools. From an equity perspective, government has 
helped charter schools meet their obligations of providing equal access to all and using 
public funds responsibly. 

State authorizers in Massachusetts and Texas still have significant work to do to create 
the tools, and more importantly, the culture necessary to adequately measure and 
improve school performance, but they have made significant strides toward defining 
accountability anew. The work of these authorizers runs counter to the compliance 
focus of how "accountability" has been defined in the past. Rather than just ensuring 
that a particular approach is in place, these authorizers needed to make subjective 
judgments about the quality of applications and the ongoing success of a given charter 
school. Authorizers need to continually make strides to improve the transparency of the 
process so that all stakeholders know what is expected and how performance will be 
measured. The TAAS system with its clear benchmarks and well-developed web-site, is 
a step in this direction. The test of that system will be in whether schools will experience 
real consequences for not meeting those expectations. To date, this is still a challenge in 
Massachusetts and Texas. ^ 

Lesson 5: Capacity is the Neglected Component of Accountability 
If schools are to improve, they not only need the incentive to improve and the flexibility 
to respond to that pressure, they also need the capacity to improve. The schools in my 
sample had significant organizational needs ranging from facilities, financing to teacher 
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professional development. Too often accountability is seen as a three-legged stool of 
standards, measures and consequences. Understanding the need to address the capacity 
issues of schools, and moreover, how oversight agencies were going to meet those needs 
was underdeveloped in both of the states studied. 

The Massachusetts and Texas authorizers did not have the fiscal and human resources to 
support the financial and technical needs of charter schools in their states. The state 
agencies provide the primary financial support (roughly 87%) through basic student 
tuition payments. However, these funds generally fell short of addressing the financial 
demands of these independent public schools. Therefore, developing a strategy to fully 
leverage and mobilize all of the public and private resources at their disposal will be 
critical to the large-scale success of their respective initiatives. 



ERIC 



Lesson 6: Incentives Should be Built on Customized and Comparable Data 
In choosing an incentive structure, authorizers face a tradeoff in terms of how much to 
rely on site visits versus state test scores as a means of measuring school performance. 
Site visits as designed in Massachusetts provided excellent formative data, elicited 
strong teacher buy-in, and facilitated the growth of organizational capacity among all 
the schools in my sample, i.e., helped schools improve their instruction, leadership, and 
use of resources. However, these site visits were limited as high stakes summative 
evaluation tools and were much more expensive than paper and pencil tests. 

State tests are much more reliable and valid than site visits as summative evaluations, 
and are relatively inexpensive, time-efficient measures of performance. Their benefit as 
a means of building organizational capacity are limited in that only the lowest 
performers in my sample responded positively. Moreover, the analysis of test score data 
in these two states still emphasized absolute performance levels over measurements 
over time, i.e., value-added models, so the power of this data was not fully tapped. In 
short, paper and pencil tests address the need for a relatively inexpensive and valid 
measure and they may inspire lower performing schools to improve. However, if the 
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goal is to push all schools to engage in a continuous improvement process, than a 
thoughtful site visit process is an essential compliment. 



Despite the strides being made in gathering good data, the charter school authorizers in 
my study both had difficulty in making high stakes decisions based on the data that had 
been collected. A contributing factor in this challenge is that the process for establishing 
the initial contracts for performance, e.g., accountabilit)^ plans, could be substantially 
improved. Being able to mete out consequences based on performance is contingent 
upon clear expectations. When those expectations are muddy, enforcing consequences 
will be difficult. Transparency should be a goal throughout the oversight process. 

In sum, all of these components — politics, opportunity, capacity, and incentives — are 
all a piece of the same puzzle. A political climate that supports decentralization 
provides schools with significant freedom or opportunity, but it also serves to stretch 
their organizational capacity. This finite capacity may in turn limit the effectiveness of 
an incentive structure designed to push schools to improve. If policymakers are going 
to improve systems of schools, accountability needs to be understood as something that 
is more complex than the three-legged stool of standards, measures, and consequences. 
My hope is that this analysis will inform the work of charter school authorizers as well 
as policymakers at the state and district levels exploring this basic bargain of autonomy 
for accountability. 

IMPLICATIONS 

Policymakers across the United States will be asking questions about opportunity, 
capacity, and incentives as public education continues to move toward decentralization. 
I agree with Bruce Fuller (2000) when he states that education reform is "not a zero sum 
game" (p.27) - the solution is not, as advocates on either end of the political spectrum 
assert, complete deregulation or absolute reliance on bureaucratic monopolies. 
Government has a role to play in the oversight of charter schools and public schools in 
general. Policymakers are striving to find an appropriate balance between Tyack's "one 
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best system" and letting a thousand flowers bloom; between excellence and equity; 
between the individual and the social good. 



An Incremental Approach to Increased Decentralization 

There are two lines of thought in terms of how decentralization might become more 
prominent in US public education. One approach is incremental and the other is more 
radical. The No Child Left Behind Act of 2002 (NCLB) may expedite the incremental 
approach. Some of the key components of the plan call for state testing in math and 
reading at grades 3-8 and 10-12 for all schools, and if the schools do not meet state 
specified expectations for adequate yearly progress for two consecutive years, the state 
is required to remediate. If this support fails to produce improved performance, the 
school becomes eligible for reconstitution. NCLB reflects the core principles of President 
Bush's education agenda and the crux of the basic bargain: "accountability, flexibility, 
local control, and more choices for parents" (Schlessinger, 2001, p. Al). Like the 
bipartisan support evident in the Massachusetts and Texas charter legislation. 
Democratic Senator Edward M. Kennedy emphasized, "This wasn't a Republican 
education program, this wasn't a Democratic education program, it's an education 
program (Schlessinger, 2001, Al). If history is a reasonable predictor of the future, then 
this policy will likely have a significant impact on how state and local agencies govern 
public education.2 State and local agencies will need to decide how to implement the 
broad intents of the law. In essence, they will need to discern the appropriate amount of 
flexibility to provide local schools, the type and form of public support to provide failing 
schools, and how best to measure school performance. 

A More Radical Approach to Decentralization: Charter Districts and Contracting 
A more radical path to decentralization is the charter district or contracting approach. 
Charter districts could have many of the benefits of the charter school model — choice, 
local control, and performance-based accountability — while addressing some of the 
challenges of the current charter schools.^ Depending on the local context, schools could 
have control of fiscal and human resource decisions and district offices might have the 
primary responsibility for evaluating school performance. Parents could have the 
power to choose their child's school, but the districts, rather than the schools, would 
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likely be the local education agency. Districts could give the schools access to facilities 
capital and financing, perhaps the greatest challenge to independent charter schools, 
and schools would have the option of utilizing the economies of scale available through 
a district structure for managing special needs services and purchasing materials and 
services. A key difference in such a district over most current district structures would 
be that the schools would establish a contract with the district defining the parameters of 
this relationship and the mutual expectations regarding performance. 

For this shift in governance to come to fruition, it will have to overcome significant 
opposition. Local school boards have had the power to give schools charter-like status 
without enabling legislation (e.g., pilot schools in Boston), so it is not surprising that 
with legislation (i.e., campus chcirters in Texas and Horace Mann charters in 
Massachusetts), districts have been slow to provide these freedoms.^ Entrenched 
stakeholders will likely resist this concept. Michael Sentence, former Massachusetts 
Secretary of Education, summarized the challenge thus: "The problem with the concept 
is that the people who hold the power do not like to share" (conversation with author, 
2001). It will not be an easy task, but transforming the culture and the functionality of 
U.S. school districts and by extension our schools could be the most important 
contribution charter schools could make to public education. 



FUTURE RESEARCH 

There are at least two areas of further research within this topic. First, there is very 
limited resecirch available on how to build capacity among independent public schools 
such as charter schools. Second, the issues surrounding how best to measure and use 
performance data cire ripe for study. In particular, research on how best to share test 
data so that is it is helpful in the continuous improvement of schools, and research on 
how best to utilize site visits in reliable and cost-efficient ways would be particularly 
beneficial. 

Endnotes 
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^ In Massachusetts, the Lynn Community Charter School was closed in February of 2002 for not 
meeting the state's expectations for academic performance. This is the first charter school in the 
state and one of the few charter schools in the nation closed for primarily academic reasons. 

2 When the Elementary and Secondary Education Act was reauthorized in 1994, the states had to 
demonstrate reasonable gains through some form of state testing in order to be eligible for federal 
Title I funds. These funds represent about 7% of the average education dollar. As of 2001, 49 
states had some form of state standards and testing system in place. 

^ See the following for discussion of the charter district concept: Education Commission of the 
States, 1999; Hassel 2001; Odden and Busch, 1998; Price, 2000. 

As of 1999, only three districts other than Houston (which had 22), opted to create campus 
charters in Texas. In Massachusetts only seven of the possible 13 Horace Mann charters were 
granted three years after the legislation was passed. 
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ATTACHMENT 1 



Interview Protocols 

Questions for Leg;islators 



General Interest 


Questions 


Roles 


♦ What was your role in the enacting the charter school law in this state? 

♦ How, if at all, have you been involved in subsequent amendments to 
the charter school law? 


Intent of charter 
law 


♦ Who were the major players involved in passing this law? 

♦ What was the intent of the charter school law from your perspective? 

♦ How will you know whether this law has met its intended purpose(s)? 

♦ How is it the charter school initiative progressing thus far? 



Questions for Authorizers 



General Interest 


Questions 


Application 

Process 


♦ First, what does it mean for your agency to authorize a charter school? 
[Ask specific questions that are not clear in the documentation 
available.] 


Oversight 


• Once yoii authorize a charter, how do you assess whether charter 

schools you authorize are successful and are meeting the terms of their 
charter? 


Technical 

Assistance/ 

Renewal 


♦ Are the charter schools you sponsor held to the same, higher lower, or 
different, standards of academic progress than district schools? 

♦ What have you learned over time about assessing charter school 
performance? 

♦ What kinds of technical support/ training do you provide your 
schools, if any? 

♦ Has your agency established any kind of criteria for revoking or not 
renewing a charter? If yes, what are they? [Probe: How good is good 
enough?/ How bad is bad enough?] 

♦ Do you think the charter schools you have sponsored have a clear 
understanding of what is expected of them in order to have their 
charter renewed? 


Capacity 


♦ How is this office governed (e.g., arm of the Board of Education)? 

♦ How many staff people at your organization work with charter 
schools? 


Politics 


♦ What are your budgetary constraints? 

♦ What are the professional backgrounds of these people? 

♦ How long have you and they been working with this organization? 

♦ How does the general public perceive charter schools? 

♦ What are your thoughts on how the charter school law was 
established here? 

♦ Do you anticipate politics playing a role in your efforts to hold these 
schools accountable? 
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Questions for Charter School Leaders 



General Interest 
Mission 



Internal 

Accountability 



External 

Accountability 



Organizational 

Capacity 



Questions 

♦ What distinguishes this school from others? [How would you 
describe it to a friend interested in sending their child to the school?] 

♦ What are the school's biggest goals met/ unmet? 

♦ How are decisions made at this school? [academic, programmatic, 
and fiscal] 

♦ What role do parents, students, governing board play in making 
decisions at the school? [Examples] 

♦ What are the roles and responsibilities of the Board [and possibly 
management company] versus your role as the director? 

♦ How are you evaluated? 

♦ How do you evaluate your teachers? [If you have fired a teacher, 
please describe the circumstances and the ramifications of that 
decision?] 

♦ How do parents know what their children are learning? [Does your 
school provide annual reports and / or comparisons to district 
performance?] 

♦ What is your relationship with your authorizing agency? 

♦ Who and how often do you see representatives from your 
authorizing agency? 

♦ What type of communication have you had with your authorizer? 

♦ [If they have received a letter or evaluative report] — What did you 
think of it? Was it helpful? What was your response? 

♦ How do you think your authorizer would define "success?" How do 
you define success? 

♦ What other forces, if any, influence how the school develops its 
policies? [community, donor, management company, accrediting 
agency, district, other government agencies, e.g. Special Education, 
state auditor] 

♦ Roughly, what percentage of your budget is spent on professional 
development? 

♦ How does that professional development focus on the school's 
goals? [Examples?] ^ 

♦ What role do teachers have in shaping school policy? What is their 
actual influence on curriculum, instruction, student assessment, and 
staff development? 

♦ To what extent do teachers work together on high quality decisions 
that affect their daily lives? (high, medium, or low) 

♦ To what extent do the faculty have a clear shared sense of purpose, 

collaborate, and share collective responsibility in the school? 
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Questions for Charter School Teachers 



General Interest Questions 



Mission 



Internal 

Accountability 



♦ What distinguishes this school from others? [How would you 
describe it to a friend interested in sending their child to the school?] 

♦ What are the school's biggest goals met/unmet? 

♦ How did you get here? Is the school what you expected? 

♦ What role do parents, students, and board members play in the 
running of this school? [asked separately] 

♦ How do you know what to teach? How much of your curriculum is 
dictated by your school design, especially if managed by a private 



External 

Accountability 



Organizational 

Capacity 



company? 

♦ How do the state standards influence what you teach? 

♦ What is your relationship with your authorizing agency? 

♦ Who and how often do you see representatives from your authorizing 
agency? 

♦ Has the school changed its practices as a result of feedback from the 
state? 

♦ How do you think your authorizer would define "success?" How do 
you know when you are successful? 

♦ Are you pursuing accreditation? What are the costs and benefits? 

♦ How is your work supported by professional development? 

♦ Is that professional development focused and sustained on the 
school's goals? [Examples?] 

♦ What role do you have in shaping school policy? What is your actual 
influence on curriculum, instruction, student assessment, and staff 
development? 

♦ To what extent do teachers work together on high quality decisions 
that affect their daily lives? (high, medium, or low) 

♦ To what extent do the faculty have a clear shared sense of purpose, 

collaborate, and share collective responsibility in the school? 



Questions for Charter School Parents 



General Interest 
Mission 



Internal 

Accountability 



External 

Accountability 



Questions 

♦ What distinguishes this school from others? [How would you 
describe it to a friend interested in sending their child to the school?] 

♦ What are the school's biggest goals met/unmet? 

♦ Why did you select this school? Is the school what you expected? 

♦ What role do parents play in the running of this school? Examples of 
policies that parents have created or influenced? 

♦ How is your child doing here? 

♦ How is the school doing generally? How do you know? 

♦ How aware are you of state perceptions regarding the school's 
performance? Have you seen any written documentation from the 
state? 

♦ The school is up for renewal this year, what is the major argument for 
the renewal of this school's charter? What evidence do you think will 
be most important to the state? 

♦ Are there any other influences, aside from the school's authorizer, 

that you believe has an influence on how the school functions? 
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Questions for Charter School Students 



General Interest 
Mission 

Internal 

Accountability 



External 

Accountability 



Questions 

♦ What distinguishes this school from others? [How would you 
describe it to a friend interested in applying to this school?] 

♦ Why did you select this school? Is the school what you expected? 

♦ What does a typical day look like for you? 

♦ WTiere did you go before and how is this school the same or different 
from your previous school? 

♦ What role do students play in the running of this school? Examples of 
policies that students have created or influenced? 

♦ How are you doing here? 

♦ Can you describe the most interesting assignment or project you have 
done this year? [Preferably, they would bring the work with them.] 

♦ Why did you choose this piece? 

♦ How do you get feedback on your work? 

♦ How is the school doing generally? How do you know? 

♦ How does the general public see your school? 

♦ The school is up for renewal this year, what is the major argument for 

the renewal of this school's charter? What evidence do you think will 
be most important to the state? 



Questions for Charter School Governing Board 



General Interest 
Mission 



Internal 

Accountability 



External 

Accountability 



Questions 

♦ What distinguishes this school from others? [How would you 
describe it to a friend interested in sending their child to the school?] 

♦ What are the school's biggest goals met/ unmet? 

♦ Who is on the board? How were you selected? Why did you choose 
to take on this responsibility? 

♦ How are decisions made at this school? [academic, programmatic, and 
fiscal] Has this process evolved over time? [e.g. survival mode to 
strategic planning?] 

♦ What role do parents and faculty play in making decisions at the 
school? [Examples] 

♦ What are the roles and responsibilities of the Board [and possibly 
management company] in relation to the director? Who makes 
hiring/ firing decisions relative to teachers? Do you need to sign off 
on the budget? 

♦ Do you evaluate the director's performance? On what basis? Can you 
fire her/him? 

♦ Please describe a difficult decision that you've made recently? 

♦ How do parents know what their children are learning? [Does your 
school provide annual reports and / or comparisons to district 
performance?] 

♦ What is your relationship with your authorizing agency? Who and 
how often do you see representatives from your authorizing agency? 

♦ What type of communication have you had with your authorizer? 
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Organizational 

Capacity 



♦ [If they have received a letter or evaluative report] -- What did you 
think of it? Was it helpful? What was your response? 

♦ How do you think your authorizer would define "success?" How do 
you define success? [Why should your charter be renewed?] 

♦ What other forces, if any, influence how the school develops its 
policies? [community, donor, management company, accrediting 
agency, district, other government agencies, e,g. Special Education, 
state auditor] 

♦ Roughly, what percentage of your budget is spent on professional 
development? 

♦ How does that professional development focus on the school’s goals? 
[Examples?] 

♦ What role do teachers play in shaping school policy? What is their 
actual influence on curriculum, instruction, student assessment, and 
staff development? 

♦ To what extent do the faculty have a clear shared sense of purpose, 

collaborate, and share collective responsibility in the school? 
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ATTACHMENT 2 



Charter School Profiles 

Overview 

I had six schools in my sample, three from Texas and three from Massachusetts. Each set of 
schools included a high, medium, and low performing school per state. 

These schools were selected based on their relative performance on state tests. Because most 
charter schools in both states were not performing exceptionally well or exceptionally poorly, this 
is not a representative sample. Further, to reduce my travel costs, once I found a school that was 
willing to participate in the study, I tried to engage at least one other school in that area. 

I chose the schools based on the following criteria: 

1 All the schools were new start-ups. (as opposed to private or public conversions): I chose start- 
ups since they were the predominant group among charters nationally (65%) (USDOE, 2000), 
and because this group of schools reported higher levels of autonomy than public school 
conversions (Celio, 2000). 

2 All the schools were in operation for four years or more. This was important for three reasons: 1) 
these school leaders participated in the USDOE charter school leader survey (USDOE 2000), 
an important context for my case studies; 2) the schools had time to build a record of 
performance, e.g., at least three years of test data and progress reports to review; and 3) these 
schools were going through the charter renewal process around the time of my visits, a 
critical test for state accountability policies. 

3 The schools represented a range of performance. Viscusi and Zechauser (1979) assert that a policy's 
strength is dependent on its ability to respond to schools with different performance levels, 
and Fuhrman (1999) points out that most of the attention of accountability systems is focused 
on the low-performing schools. Therefore, I chose a high-, low-, and medium-performing 
charter school based on state test scores in the 1997-98 academic year. For the purposes of 
this study, the high-performing charter school in Texas was labeled "TX-A," the medium- 
performing school was labeled "TX-B," and the low-performing school was labeled "TX-C". 
The three Massachusetts schools were labeled in a similar manner (MA-A, MA-B, and MA- 
C). 

4 The schools were state-sponsored (as opposed to district- or university-sponsored). This is because 
state-sponsored charter school leaders reported higher levels of autonomy than district- 
sponsored charters (Hill et al., 1999) and because policy information was more accessible 
through the state than other authorizing options such as district- or university-sponsored 
charters.* 

To tighten my focus, all site visits included interviews with English Language Arts teachers in the 
fourth and/or eighth grade (subject areas and grade levels that are tested in Massachusetts and 
Texas). 
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The profiles include basic descriptive information on each of the schools, i.e., enrollment size and 
location. Each of the profiles also summarize the demographic information, spending patterns, 
and relative state test score performance for each school. For the sake of simplicity, I have 
included test score data for a single grade in the charter school relative to its surrounding district 
and the state. 
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MA-A 



Opened: 1995 

Location: Suburban 

Grades Served (1999-2000): 7-12 

Enrollment (1999-2000): 346 

Facility: Former government facility 

Management Contract: Self-Managed 

Average Years of Teaching Experience: 6.2 years 

Rating: Not applicable 

Description: The school is based on a nationally recognized progressive 
curriculum. 



MA-A Student Demographics. This graph represents the percentages of 
students across the various student categories. The left column represents the 
charter school, the middle column represents the district average, and the right 
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MA-A Budget Allocation 




MA-A Grade 8 MCAS Scores. This graph summarizes the scale scores of 8* 
grade students in the charter school, home district and the state. 
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MA-B 



Opened: 1995 

Location: Urban 

Grades Served (1999-2000): K-11 

Enrollment (1999-2000): 688 

Facility: Leased parochial school 

Management Contract: EMO 

Average Years of Teaching Experience: 9 years 

Rating: Not applicable 

Description: The school is based on an EMO-developed college preparatory 
curriculum. 



MA-B Student Demographics. This graph represents the percentages of students 
across the various student categories. The left column represents the charter 
school, the middle column represents the district average, and the right column 
represents the state average. 
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MA-B Budget Allocation 




MA-B Grade 8 MCAS Scores. This graph summarizes the scale scores of 8* 
grade students in the charter school, home district and the state. 
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MA-C 



Opened: 1996 

Location: Urban 

Grades Served (1999-2000): K-8 

Enrollment (1999-2000): 312 

Facility: Leased parochial school 

Management Contract: Self-managed 

Average Years of Teaching Experience: 4 years 

Rating: Not applicable 

Description: The school is based on an EMO-developed college preparatory 
curriculum. 



MA-C Student Demographics. This graph represents the percentages of 
students across the various student categories. The left column represents the 
charter school, the middle column represents the district average, and the right 
column represents the state average. 
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MA-C Budget Allocation 




MA-C Grade 8 MCAS Scores. This graph summarizes the scale scores of 8* 
grade students in the charter school, home district and the state. 
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TX-A 



Opened: 1996 
Location: Suburban 
Grades Served (1999-2000): 6-10 
Enrollment (1999-2000): 409 

Facility': Two campuses: a leased parochial school and a former horse ranch 
Management Contract: Self-managed 
Average Years of Teaching Experience: 4 years 
Rating: Recognized 

Description: TX-A is a "college preparatory" school with a curricular focus 
based on the International Baccalaureate. 



TX-A Student Demographics (2000). This graph summarizes the percentage of 
students in each demographic grouping. The relative percentages of the charter 
school, its home district and the state are presented from left to right in each set 
of columns. 
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TX-A Budget Allocation (2000) 




TX-A Grade 8 TAAS Scores (2000). This graph summarizes the percentage of 8* 
grade students in the charter school, home district and the state, that have passed 
the minimum score or higher. 
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Opened: 1996 
Location: Urban 

Grades Served (1999-2000): PreK-5 
Enrollment (1999-2000): 258 
Facility: Single-story office space. 

Management Contract; Self-managed 
Average Years of Teaching Experience: 1.1 years 
Rating: Acceptable 

Description: The curricular focus of this school is to improve literacy. The 
school is predominantly made up of Latin American and Central American 
immigrants. 



TX-B Student Demographics. This graph summarizes the percentage of students 
in each demographic grouping. The relative percentages of the charter school, its 




home district and the state are presented from left to right in each set of columns. 
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TX-B Budget Allocation 




TX-B Grade 4 TAAS Scores (2000). This graph summarizes the percentage of 4* 
grade students in the charter school, home district and the state, that have passed 
the minimum score or higher. 
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Opened: 1996 
Location: Suburban 
Grades Served (1999-2000): 5-12 
Enrollment (1999-2000): 556 

Facility: The school was split across two campuses and both sites were based in 
former office space. 

Management Contract: Seif -managed 
Average Years of Teaching Experience: 2.1 years 
Rating: Low 



Description: The curricular focus was unclear, but the literature describes the 
school as technology -based and experiential. The school was closed in the fall of 
2000 . 



TX-C Student Demographics. This graph summarizes the percentage of students 
in each demographic grouping. The relative percentages of the charter school, its 
home district and the state are presented from left to right in each set of columns. 
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TX-C Budget Allocation 




TX-C Grade 8 TAAS Scores. This graph summarizes the percentage of 8* grade 
students in the charter school, home district and the state, that have passed the 
minimum score or higher. 
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^ The Center for Education Reform reports that only 5.6 percent of charter schools 
nationwide are in the 12 states requiring local school boards to approve charter school 
applications, and within those states there is an average of 9.6 charter schools per state. 
In contrast, states with multiple chartering authorities or a strong appeals process (e.g., 
California, Arizona, and Michigan), have an average of 80.8 charters per state. (For more 
data on the reduced rate of charter granting within districts, also see Wells et al., 1998, 
and Hill et al, 1999.) 
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The following is divided into four parts. First, I provide my scoring rubric. Second, I provide an example of how I scored a school. Third, I 
describe the scoring process. Fourth, I summarize my assumptions for each of the criteria. The summary scores for all schools are provided i 
Table 6C. 



Criteria 1. Teacher Professional Knowledge and Skills 
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Criteria 2. Effective Leadership 
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This turnover rate relates to member changes that are not related to the planned turnover associated with the term limits established in a board’s by-laws. 



Criteria 3 . Fiscal and Curricular Autonomy 
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Scoring the Rubric 

When scoring each school on a given component (e.g., teacher knowledge and skills), I looked across the rows of each dimension and ranked the 
school from "1" (poor) to "3" (strong) (see Criterion 1). I then looked down the score column and developed a single holistic or overall score for 
each criterion.2 For example, while the component of "teacher knowledge and skills" has four dimensions, the overall score does not represent an 
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^ While these scores were inherently subjective and thus run the risk of being challenged in terms of their validity, since I conducted 
all the work myself the scores should be consistent or reliable. 

^ Teacher certification was not included in this dimension because it is not clear that certified teachers are consistently better prepared than non-certified teachers. 
In fact, some charter schools aroimd the coimtry have successfully relied upon mid-career professionals to provide important contributions to their schools. 

^ Bruce Cooper as reported in Mandell, 1996, foimd that 1.5% of public school spending. 
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Curricular and fiscal autonomy was easier to define. I examined just two dimensions in this component. 
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Criteria 1. Teacher Professional Knowledge and Skills 
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Criteria 2. Effective Leadership 
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This turnover rate relates to member changes that are not related to the planned turnover associated with the term limits established in a board’s by-laws. 



Criteria 3. Fiscal and Curricular Autonomy 
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^ While these scores were inherently subjective and thus run the risk of being challenged in terms of their validity, since I conducted 
all the work myself the scores should be consistent or reliable. 

^ Teacher certification was not included in this dimension because it is not clear that certified teachers are consistently better prepared than non-certified teachers. 
In fact, some charter schools around the country have successfully relied upon mid-career professionals to provide important contributions to their schools. 

^ Bruce Cooper as reported in Mandell, 1996, found that 1.5% of public school spending. 



o 

o 



<u 

> 

QJ 



CQ 

(0 



QJ 

X 

U 

(C 

QJ 



U 

OJ 



o u 



cn 



cn 



.& CO 

X 'o 

Si 

O) 



I s 
I s 

S :s 
'5 ^ 
S) ^ 
S « 

QJ 

X 



(A 



T! 'w ?i -ti 6 



O 

a. 

ra w 

4-1 (C3 

5 c 

:S .2 

QJ S 

C (U 

0 o 

m iH 
(C Q. 

^ M) 

::: c 
o 

CO j3 
"O CO 

^ C 
0 ) ns 

^ a> 

QJ (-1 

1 « 
o 

ns CO 
C >-. 

.2 'S 
S :S 

o> 

'o ® 

o ^ 
C 

0 S 

s - 

4-1 

'S ^ 

> a 

at ^ 

>-i o 
■£ §) 

JCh qj 

« o 

ii § 

“ 5 

■a -c 

c ^ 

1 o 

at 

"T3 at 
V C 

i'= 

3 0 ) 
CO a. 

at 



X 
'co 
- QJ 

^ ■£ 
at 

o 

2 ^ 

u 

ns QJ 

1 ■" 

o 6 

VM 

^ XI 

4-. 3 

o o 
C c 
QJ at 

S -o 

2 S 

« rs 



QJ O 

S ^ 

x: QJ 

I " 



^ o 

> -o 

.2 

QJ ns 

I ^ 

c :2 

C TJ 



E 



QJ 



CO ^ 



ns QJ 
.2 > 
5 ^ 

"S O 
V X 



- c 
3 ^ 
O TS 



2 03 

^ (-1 



<*4 . V J 

2 (o :;2 

C QJ "O 

.2 '-n at 

4-> >-l 

#« rti 



-o £ 



o 

a. 



CO O 9 43 



6 

at 

U4 

ns 



as ^ 
as CO 

d. a. 



CO 

C 

O 

a.- 

CO : 

a ■. 



at )H 

^ ns 
C QJ 

« ’i: 

C C 
ns at 



O 

CQ ^ 

W QJ 

< .2 
. 3 .c 

QJ CT os 
c QJ b 

QJ CO CO 

n ^ 3 

° S I 

^ s 



a. 

n ^ 
S »H 

33 os 
T3 at 
C ^ 

— • QJ 

Bt 

o o 

CO ^ 



|i:S 



3 03 

u x: 

CI 2 U 



"T3 



c 
at 

QJ »-4 

Si 
6 ^ 

^ *15 
•S'X 
^ H 

J c 
.2 
• S 13 

u 3 
±! 



at 

6 

3 

CO 

CO 

ns 

-o 

u 

ns 

O 

jD 



at 

x: 



c 

ns 

crt XZ 
CO *3 

^ CO 

i 8 

^ -S 

QJ 



QJ 

> 

QJ 

CO 

bO 

C 

o 



c 

o 

04 



QJ 

-C 



"O 

at 

-o 
c 

0 X 
04.ti 
CO > 
QJ > 

^ c 

,2 o 

03 *X3 
O- U 

1 i 

S- 8 

at 
x: 



ns 
X 
at 

QJ I— I 
> 



c 

o 

E 
E 

i ^8 
-c -S ^ 

■*^ Jh C 

ns 



C 

ns 

33 



ns 

*4 ^ 



.ti QJ 

CJ ^ 

5 QJ QJ 

.S 

^83 

ns ^ "O 
^ ” o 

- -a '-’ 

03 2 

.a S 9 

q ^ c 

S QJ .2 

^ E z: 

o 

CO 



C 

o 



o 

CO 



CO 



(J 



QJ bp c 

d ^ 

ns CO ^ 

-c g ^ 

CO .2 QJ 

QJ 33 3 

2 N os 

^4 C 

at ns ^ 
> 
o 
c 



bO > 

(-1 

o 



(-1 

nS 

QJ 

>^ 



5 .2 
5 i' ^ 



O 4 



.9 S “ 

U4 -C 

o. -a 
t-* c 

i “ 

ns 



(-1 

at 

> 

o 

bO 

c 



JC 

J3 



o 

o 

j: 

CJ 

CO 

4-> 

ns 

x: 



U4 03 
Cl '*“’ 

O “ 
CL -5 



QJ 

Sd 



QJ 
> 
QJ 

CO ‘‘^ O 

S 9 K 



(U 
6 
a a 



ns 



b "O 

« QJ 
""CLCC 
^ C 



2 “ g 



cu 



— i-i 

2 



! I 

^ s 

o u 



V QJ 

>> x: 



bO 

c 

*N 



4J^ QJ 
= 2 - 
*> 2 
^ c 'S 

CO .t3 o 

QJ O 

'2 ^ ^ 

^ ^ o 

Iq CJ^ *- 

'b as ^ 

CO ON 13 
r- _i 



CLP ^ 

5 u -s 



at 3 

i .g £ 



5 03 

O QJ 

x: x: 



u 

(« 

o 

Li ' 

CL 

CL 

ns 



QS QJ • 

• B“ > ■ 
c c •: 



§ s 



c . . ^ 
o dr ^ 

^ -8 
2 q 3 g 

5 5 



ns 



O ^ 



O 4 ^ 



tsO 

< -S 

. V4-I 

CO ’3 
QJ 4C 
CO 
L 

13 o 

33 u 

^ s 

QJ ^ 
CO y 



ns 

x: 

4-1 

CO 

"O 

Li 

ns 

o 

X^ 

bO 

C 



at 

> 

o 

bO 

at 

E 

« 



QJ .Xi 

!3 

r* 

« § 

O 

C QJ 
n ^ 

O 4-1 

"O VM 

^ O 
eg QJ 
*4 'o 

CO ^ 

T3 QJ 

(C 4-P 

O M-i 

§ .fr 

JC 



.2 o 

CJ 

ns ns 



« ^ 



os 

o 

bo , 
C ' 



§i 

■8 I 

CO 3 

« o 

bO o 

.g Id 

s = 

c o 



CL 

O 

bo 



C os 

(Q 4-1 

X c 
CJ at 



QJ 

13 

ns 

C 

QJ 

> 



QJ 

*35 

C 

o 

CL 



nS 

QJ 



.2 .2 « 
J3 X ^ 



CO ^ 



.IT o 

fC 

CO ns 
^ -t3 

-S .2 

s .§ 



^ CL 

ns ^ 
u ^ 

O CO 
H 



13 



13 

” i 

.2 M 

CO QJ 
> 2 

-C ^ 

r ^ 

ns QJ 
XZ bO 

-M 13 

at ns 

E ^ 

3 Q 



os Oi 

m I 
8i 

I i 

L O 
CL 

L QJ 

ns 

^ .t: 

CJ S 

VLi *3 

O > _; 
C 2 o 

o o 
-9 CO X 

"rt CJ 
y CO 

ns 



:33 > 



(0 - 
'o £ 

CO ^ . 

(/5 L 

■9 c 
> ,0 
lo 



ns 



(b 

F — 4 >-H 

QJ QJ . 

Jh -*-* 

QJ C 

*2 

-t-p ►JL ^ 

s 

> 2 

U £ 



g P 



b 

CO 

13 

C 

P 



^ o 

4*- M-l 

■ti QJ 

s 



CO .3 

at 



c bO 

il 

ns 



a; -a 



bO 

C 



ns 



at 

« CO S 
u 



CO .3 
Q 4 



O fC 
CO Li LL 

QJ QJ -n 

04 2 



ns 

X 



f£j 4-p 



bO ^ 

c E 



^ « 
b QJ 

s g 

c S 

d 04 
S X 

QJ 

.2 CO 

4-1 

- 

o g 
X d 
bO o 
C o 
•£ X 
C CJ 

S 

o -£ 
bo d 
ns 2 
O 

X QJ 

X L 

O 3 

CO 

c .S 

•2 

Li 33 

O S 

CL O 



bO 

c 

'W) 

bO 

P 

ts 

CO 

QJ 



QJ 



"§ 'qj 

^ s I 

F§ « 

' bi ^ ^ 

CL O X 
■£ X ii 
QJ at ns v*_ 2 QJ 
rs K t-i ^ CO CO 

CO « 

^ 2 'C 



o 

.. o 

QJ X 



lb CL CO 



bO CO 
ns 13 






P C 



CO 

C 
o 
X is 
<b s: 

44 w 

CJ 

^1 

QJ 



^ t: 

^20 
nS _Q V 44 

ri M-i 

d bO QJ 
^.S C 
'fl g 2 
QJ at § 

C > E 

at o o 

bO bO u 



CL ^ _ 

8-0 

S g o 

-T3 t; c 

s: o -t 3 
« ^ c 
s: E ^ 

O X M- 

:s 2 *13 

S -S5 



c o 

d c 

§ 2 

2 P 
c o 

X ^ 

44 4Q 

^ o 
ns o 

I 3 M 
< 8 
^ J3 

« « 
CCS 

bo X 
IS 2 

3 •£ 
-Q ^ 

I 6 

« P 

*3 CO 

rS CJ^ 
QQ ns 



ex 

X 

QJ 

CO 

ns 

QJ 

CO 

P 

ns 

u 

QJ 

X 



P 

X 

CJ 

ns 



at 

o 

X 



CO QJ 

1 ^ 

S S) 

P o 
ns 

U CO 
QJ -44 

X QJ 

n p 

CN 2 

^ p 

QJ C 

bO CO 

2 2 
at — 

g £ 

If 

4- C 

at QJ 

I.e 

C at 
. 2 ^ 
2 S' 

M§ 

.E ^ 

X 44 ' 

c o 
QJ X 
O 4 X 

c ^ 

I i 

4-> 3 

ns 

i C 
CO ns 

II 



QJ 

X 



X X 

r- ON 
5 ON 



CJ X 



52 2 

•£ <N 
at 4 - 
X 



X -t! iS ^ 



ns C 
X ^ 
QJ at 



s 

at 
d X 

o o 

o ^ 

'S CO 

g X 

t: 2 

ns ns 

'S *> 

O 33 

t: 

C CL 

2 'qj 

o 

^ a' 

.§ g 



bO 

c 

X 

ns 

CJ 

O 



ns ns 
2 CO 

CO ns — <■ 

s "S s 

X 5 fb 
S X ^ 

^ ja X 

be X Cl 
o o .er 
2 o X 

= •5 2 
•a « aj 

"tJ Cl ^ 

ir at cb 
S X QJ 

L 1 -^ 

I 0 2 QJ 
K X 

*— 4 CJ 4-P 



CO 

QJ 



c 

o 

CL 



c 

ns 



ns 

QJ 



X 

Li 

ns 

O 

X 



P 

b 



at 

X 

ns 

at 



X 

ns 



X *0 

fL; r« 



ns 

X 

44 

at 

C 

o 



CO 

QJ 

U 

(H 

P 

o 

CO 

QJ 



2 * ^ 
X 

CO QJ 

JrJ CO 
2 P 
ns X 
" 3 

QJ O 
> ^ 
X X 
^ c 
2 § 



at 



>> CO 

F-4 — 

I C 

■g 9 

CO E 
ns d 

X ^04 

I -8 

— n 
at 2 



P ns 

CO QJ 

C X 



o 

ERIC 
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